Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Error format support, and JSON output option #11396

Merged
merged 48 commits into from
May 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
393820c
Add -O/--output CLI option
tusharsadhwani Oct 24, 2021
282bd28
Initial formatter setup
tusharsadhwani Oct 25, 2021
ccda5b0
Make error_formatter an optional argument
tusharsadhwani Oct 27, 2021
c849a77
Fix type annotation
tusharsadhwani Oct 27, 2021
fd2feab
Fix whitespace
tusharsadhwani Oct 27, 2021
b188001
Remove whitespace
tusharsadhwani Oct 27, 2021
51c1acc
Merge branch 'master' of https://github.com/tusharsadhwani/mypy into …
tusharsadhwani Oct 27, 2021
9177dab
Merge branch 'python:master' into output-json
tushar-deepsource Jan 19, 2022
bc5ceac
Add hint property to errors
tushar-deepsource Jan 19, 2022
9d29ab0
Fix lint issues
tusharsadhwani Jan 19, 2022
bd6d48d
Merge branch 'master' into output-json
tusharsadhwani Feb 22, 2023
ba8d17f
Fix import and typing issues
tusharsadhwani Feb 22, 2023
a2bc04d
Fix error tuple signature
tusharsadhwani Feb 22, 2023
35974e4
Import Optional
tusharsadhwani Feb 23, 2023
1e5ec91
Run black
tusharsadhwani Feb 23, 2023
723219f
Run black on another file
tusharsadhwani Feb 23, 2023
2228c0a
Run isort
tusharsadhwani Feb 23, 2023
33d81b0
Run isort on build.py
tusharsadhwani Feb 23, 2023
63001ea
Merge branch 'master' into output-json
tusharsadhwani Apr 19, 2023
d27be7e
Merge branch 'master' into output-json
tusharsadhwani Apr 20, 2023
efe5c5d
Merge branch 'master' into output-json
tusharsadhwani Apr 27, 2023
1872ae6
Add tests for json output
tusharsadhwani Apr 27, 2023
3abc9cb
Suggestions from code review, and negative test
tusharsadhwani Apr 27, 2023
6c9ab11
Add default value of None
tusharsadhwani Apr 27, 2023
627ed8e
Default output to None in options as well
tusharsadhwani Apr 27, 2023
e425cbe
Fix failing tests
tusharsadhwani Apr 27, 2023
47f1b07
improve docstring
tusharsadhwani Apr 27, 2023
e00ad4a
type cast
tusharsadhwani Apr 27, 2023
c1fb6a2
Another explicit type cast
tusharsadhwani Apr 27, 2023
aafe3aa
remove unused import
tusharsadhwani Apr 27, 2023
fae3215
create formatter object
tusharsadhwani Apr 27, 2023
89ad1d3
Add custom end to end test
tusharsadhwani Apr 27, 2023
7a3f736
unused import
tusharsadhwani Apr 27, 2023
6d46f75
trailing whitespace
tusharsadhwani Apr 27, 2023
e71a372
try fixing windows
tusharsadhwani Apr 28, 2023
8cca203
fix windows separator issue
tusharsadhwani Apr 28, 2023
79e16a8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 28, 2023
8bf4890
unused import
tusharsadhwani Apr 28, 2023
5899f26
Merge branch 'master' into output-json
tusharsadhwani Apr 28, 2023
880b8f3
Merge branch 'master' into output-json
tusharsadhwani May 5, 2023
0aafadf
Pass error tuples to format_messages
tusharsadhwani May 10, 2023
4cab249
Merge branch 'master' into output-json
tusharsadhwani May 10, 2023
7fe71c3
Merge branch 'master' into output-json
tusharsadhwani May 13, 2023
ad8f1d6
Merge branch 'master' into output-json
tusharsadhwani May 9, 2024
e2fd45e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 9, 2024
4b03c5c
ruff lints
tusharsadhwani May 9, 2024
e0e6896
address comments
tusharsadhwani May 10, 2024
a0dc6d1
use severity
tusharsadhwani May 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions mypy/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@

import mypy.semanal_main
from mypy.checker import TypeChecker
from mypy.error_formatter import OUTPUT_CHOICES, ErrorFormatter
from mypy.errors import CompileError, ErrorInfo, Errors, report_internal_error
from mypy.graph_utils import prepare_sccs, strongly_connected_components, topsort
from mypy.indirection import TypeIndirectionVisitor
Expand Down Expand Up @@ -253,6 +254,7 @@ def _build(
plugin=plugin,
plugins_snapshot=snapshot,
errors=errors,
error_formatter=None if options.output is None else OUTPUT_CHOICES.get(options.output),
flush_errors=flush_errors,
fscache=fscache,
stdout=stdout,
Expand Down Expand Up @@ -607,6 +609,7 @@ def __init__(
fscache: FileSystemCache,
stdout: TextIO,
stderr: TextIO,
error_formatter: ErrorFormatter | None = None,
) -> None:
self.stats: dict[str, Any] = {} # Values are ints or floats
self.stdout = stdout
Expand All @@ -615,6 +618,7 @@ def __init__(
self.data_dir = data_dir
self.errors = errors
self.errors.set_ignore_prefix(ignore_prefix)
self.error_formatter = error_formatter
self.search_paths = search_paths
self.source_set = source_set
self.reports = reports
Expand Down Expand Up @@ -3463,11 +3467,8 @@ def process_stale_scc(graph: Graph, scc: list[str], manager: BuildManager) -> No
for id in stale:
graph[id].transitive_error = True
for id in stale:
manager.flush_errors(
manager.errors.simplify_path(graph[id].xpath),
manager.errors.file_messages(graph[id].xpath),
False,
)
errors = manager.errors.file_messages(graph[id].xpath, formatter=manager.error_formatter)
manager.flush_errors(manager.errors.simplify_path(graph[id].xpath), errors, False)
graph[id].write_cache()
graph[id].mark_as_rechecked()

Expand Down
37 changes: 37 additions & 0 deletions mypy/error_formatter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
"""Defines the different custom formats in which mypy can output."""

import json
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING

if TYPE_CHECKING:
from mypy.errors import MypyError


class ErrorFormatter(ABC):
"""Base class to define how errors are formatted before being printed."""

@abstractmethod
def report_error(self, error: "MypyError") -> str:
raise NotImplementedError


class JSONFormatter(ErrorFormatter):
"""Formatter for basic JSON output format."""

def report_error(self, error: "MypyError") -> str:
"""Prints out the errors as simple, static JSON lines."""
return json.dumps(
{
"file": error.file_path,
"line": error.line,
"column": error.column,
"message": error.message,
"hint": None if len(error.hints) == 0 else "\n".join(error.hints),
"code": None if error.errorcode is None else error.errorcode.code,
"severity": error.severity,
}
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module should define a dictionary str -> ErrorFormatter that can be used in build.py.



OUTPUT_CHOICES = {"json": JSONFormatter()}
75 changes: 68 additions & 7 deletions mypy/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from typing_extensions import Literal, TypeAlias as _TypeAlias

from mypy import errorcodes as codes
from mypy.error_formatter import ErrorFormatter
from mypy.errorcodes import IMPORT, IMPORT_NOT_FOUND, IMPORT_UNTYPED, ErrorCode, mypy_error_codes
from mypy.message_registry import ErrorMessage
from mypy.options import Options
Expand Down Expand Up @@ -834,7 +835,7 @@ def raise_error(self, use_stdout: bool = True) -> NoReturn:
)

def format_messages(
self, error_info: list[ErrorInfo], source_lines: list[str] | None
self, error_tuples: list[ErrorTuple], source_lines: list[str] | None
) -> list[str]:
"""Return a string list that represents the error messages.

Expand All @@ -843,9 +844,6 @@ def format_messages(
severity 'error').
"""
a: list[str] = []
error_info = [info for info in error_info if not info.hidden]
errors = self.render_messages(self.sort_messages(error_info))
errors = self.remove_duplicates(errors)
for (
file,
line,
Expand All @@ -856,7 +854,7 @@ def format_messages(
message,
allow_dups,
code,
) in errors:
) in error_tuples:
s = ""
if file is not None:
if self.options.show_column_numbers and line >= 0 and column >= 0:
Expand Down Expand Up @@ -901,18 +899,28 @@ def format_messages(
a.append(" " * (DEFAULT_SOURCE_OFFSET + column) + marker)
return a

def file_messages(self, path: str) -> list[str]:
def file_messages(self, path: str, formatter: ErrorFormatter | None = None) -> list[str]:
"""Return a string list of new error messages from a given file.

Use a form suitable for displaying to the user.
"""
if path not in self.error_info_map:
return []

error_info = self.error_info_map[path]
error_info = [info for info in error_info if not info.hidden]
error_tuples = self.render_messages(self.sort_messages(error_info))
error_tuples = self.remove_duplicates(error_tuples)

if formatter is not None:
errors = create_errors(error_tuples)
return [formatter.report_error(err) for err in errors]

self.flushed_files.add(path)
source_lines = None
if self.options.pretty and self.read_source:
source_lines = self.read_source(path)
return self.format_messages(self.error_info_map[path], source_lines)
return self.format_messages(error_tuples, source_lines)

def new_messages(self) -> list[str]:
"""Return a string list of new error messages.
Expand Down Expand Up @@ -1278,3 +1286,56 @@ def report_internal_error(
# Exit. The caller has nothing more to say.
# We use exit code 2 to signal that this is no ordinary error.
raise SystemExit(2)


class MypyError:
def __init__(
self,
file_path: str,
line: int,
column: int,
message: str,
errorcode: ErrorCode | None,
severity: Literal["error", "note"],
) -> None:
self.file_path = file_path
self.line = line
self.column = column
self.message = message
self.errorcode = errorcode
self.severity = severity
self.hints: list[str] = []


# (file_path, line, column)
_ErrorLocation = Tuple[str, int, int]


def create_errors(error_tuples: list[ErrorTuple]) -> list[MypyError]:
errors: list[MypyError] = []
latest_error_at_location: dict[_ErrorLocation, MypyError] = {}

for error_tuple in error_tuples:
file_path, line, column, _, _, severity, message, _, errorcode = error_tuple
if file_path is None:
continue

assert severity in ("error", "note")
if severity == "note":
error_location = (file_path, line, column)
error = latest_error_at_location.get(error_location)
if error is None:
# This is purely a note, with no error correlated to it
error = MypyError(file_path, line, column, message, errorcode, severity="note")
errors.append(error)
continue

error.hints.append(message)

else:
error = MypyError(file_path, line, column, message, errorcode, severity="error")
errors.append(error)
error_location = (file_path, line, column)
latest_error_at_location[error_location] = error

return errors
17 changes: 15 additions & 2 deletions mypy/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
parse_version,
validate_package_allow_list,
)
from mypy.error_formatter import OUTPUT_CHOICES
from mypy.errorcodes import error_codes
from mypy.errors import CompileError
from mypy.find_sources import InvalidSourceList, create_source_list
Expand Down Expand Up @@ -72,7 +73,9 @@ def main(
if clean_exit:
options.fast_exit = False

formatter = util.FancyFormatter(stdout, stderr, options.hide_error_codes)
formatter = util.FancyFormatter(
stdout, stderr, options.hide_error_codes, hide_success=bool(options.output)
)

if options.install_types and (stdout is not sys.stdout or stderr is not sys.stderr):
# Since --install-types performs user input, we want regular stdout and stderr.
Expand Down Expand Up @@ -156,7 +159,9 @@ def run_build(
stdout: TextIO,
stderr: TextIO,
) -> tuple[build.BuildResult | None, list[str], bool]:
formatter = util.FancyFormatter(stdout, stderr, options.hide_error_codes)
formatter = util.FancyFormatter(
stdout, stderr, options.hide_error_codes, hide_success=bool(options.output)
)

messages = []
messages_by_file = defaultdict(list)
Expand Down Expand Up @@ -525,6 +530,14 @@ def add_invertible_flag(
stdout=stdout,
)

general_group.add_argument(
"-O",
"--output",
metavar="FORMAT",
help="Set a custom output format",
choices=OUTPUT_CHOICES,
)

config_group = parser.add_argument_group(
title="Config file",
description="Use a config file instead of command line arguments. "
Expand Down
4 changes: 3 additions & 1 deletion mypy/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -376,10 +376,12 @@ def __init__(self) -> None:

self.disable_bytearray_promotion = False
self.disable_memoryview_promotion = False

self.force_uppercase_builtins = False
self.force_union_syntax = False

# Sets custom output format
self.output: str | None = None

def use_lowercase_names(self) -> bool:
if self.python_version >= (3, 9):
return not self.force_uppercase_builtins
Expand Down
58 changes: 58 additions & 0 deletions mypy/test/testoutput.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
"""Test cases for `--output=json`.

These cannot be run by the usual unit test runner because of the backslashes in
the output, which get normalized to forward slashes by the test suite on Windows.
"""

from __future__ import annotations

import os
import os.path

from mypy import api
from mypy.defaults import PYTHON3_VERSION
from mypy.test.config import test_temp_dir
from mypy.test.data import DataDrivenTestCase, DataSuite


class OutputJSONsuite(DataSuite):
files = ["outputjson.test"]

def run_case(self, testcase: DataDrivenTestCase) -> None:
test_output_json(testcase)


def test_output_json(testcase: DataDrivenTestCase) -> None:
"""Runs Mypy in a subprocess, and ensures that `--output=json` works as intended."""
mypy_cmdline = ["--output=json"]
mypy_cmdline.append(f"--python-version={'.'.join(map(str, PYTHON3_VERSION))}")

# Write the program to a file.
program_path = os.path.join(test_temp_dir, "main")
mypy_cmdline.append(program_path)
with open(program_path, "w", encoding="utf8") as file:
for s in testcase.input:
file.write(f"{s}\n")

output = []
# Type check the program.
out, err, returncode = api.run(mypy_cmdline)
# split lines, remove newlines, and remove directory of test case
for line in (out + err).rstrip("\n").splitlines():
if line.startswith(test_temp_dir + os.sep):
output.append(line[len(test_temp_dir + os.sep) :].rstrip("\r\n"))
else:
output.append(line.rstrip("\r\n"))

if returncode > 1:
output.append("!!! Mypy crashed !!!")

# Remove temp file.
os.remove(program_path)

# JSON encodes every `\` character into `\\`, so we need to remove `\\` from windows paths
# and `/` from POSIX paths
json_os_separator = os.sep.replace("\\", "\\\\")
normalized_output = [line.replace(test_temp_dir + json_os_separator, "") for line in output]

assert normalized_output == testcase.output
9 changes: 8 additions & 1 deletion mypy/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -563,8 +563,12 @@ class FancyFormatter:
This currently only works on Linux and Mac.
"""

def __init__(self, f_out: IO[str], f_err: IO[str], hide_error_codes: bool) -> None:
def __init__(
self, f_out: IO[str], f_err: IO[str], hide_error_codes: bool, hide_success: bool = False
) -> None:
self.hide_error_codes = hide_error_codes
self.hide_success = hide_success

# Check if we are in a human-facing terminal on a supported platform.
if sys.platform not in ("linux", "darwin", "win32", "emscripten"):
self.dummy_term = True
Expand Down Expand Up @@ -793,6 +797,9 @@ def format_success(self, n_sources: int, use_color: bool = True) -> str:
n_sources is total number of files passed directly on command line,
i.e. excluding stubs and followed imports.
"""
if self.hide_success:
return ""

msg = f"Success: no issues found in {n_sources} source file{plural_s(n_sources)}"
if not use_color:
return msg
Expand Down
44 changes: 44 additions & 0 deletions test-data/unit/outputjson.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
-- Test cases for `--output=json`.
-- These cannot be run by the usual unit test runner because of the backslashes
-- in the output, which get normalized to forward slashes by the test suite on
-- Windows.

[case testOutputJsonNoIssues]
# flags: --output=json
def foo() -> None:
pass

foo()
[out]

[case testOutputJsonSimple]
# flags: --output=json
def foo() -> None:
pass

foo(1)
[out]
{"file": "main", "line": 5, "column": 0, "message": "Too many arguments for \"foo\"", "hint": null, "code": "call-arg", "severity": "error"}

[case testOutputJsonWithHint]
# flags: --output=json
from typing import Optional, overload

@overload
def foo() -> None: ...
@overload
def foo(x: int) -> None: ...

def foo(x: Optional[int] = None) -> None:
...

reveal_type(foo)

foo('42')

def bar() -> None: ...
bar('42')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a test case with reveal_type(); let's make sure that shows up in the output.

[out]
{"file": "main", "line": 12, "column": 12, "message": "Revealed type is \"Overload(def (), def (x: builtins.int))\"", "hint": null, "code": "misc", "severity": "note"}
{"file": "main", "line": 14, "column": 0, "message": "No overload variant of \"foo\" matches argument type \"str\"", "hint": "Possible overload variants:\n def foo() -> None\n def foo(x: int) -> None", "code": "call-overload", "severity": "error"}
{"file": "main", "line": 17, "column": 0, "message": "Too many arguments for \"bar\"", "hint": null, "code": "call-arg", "severity": "error"}
Loading