Skip to content
This repository has been archived by the owner on Aug 21, 2024. It is now read-only.

Troubleshooting failing transformer tests is difficult #647

Open
major opened this issue Jul 10, 2023 · 0 comments
Open

Troubleshooting failing transformer tests is difficult #647

major opened this issue Jul 10, 2023 · 0 comments

Comments

@major
Copy link
Member

major commented Jul 10, 2023

I was trying to go through and simplify some code, but I ended up with failing tests which were difficult to diagnose:

runner = <click.testing.CliRunner object at 0x7f3e770e9110>, tmp_path = PosixPath('/tmp/pytest-of-major/pytest-0/popen-gw0/test_all0')

    def test_all(runner, tmp_path):
        """Run transformer end to end and generate a list with all image
        details."""
        result = runner.invoke(
            transformer.run,
            [
                "-f",
                "tests/transformer/testdata/input/raw/google/all.json,tests/transformer/testdata/input/raw/aws/af-south-1.json,tests/transformer/testdata/input/raw/azure/eastus.json",
                "-op=.",
                f"-dp={tmp_path}",
                "--filter.until=none",
            ],
        )
    
        assert result.exit_code == 0, (
            f"expected no error, but got code {result.exit_code} and"
            f" output:\n{result.output}"
        )
    
        # Get current directory
        pwd = os.getcwd()
    
        # Check image data by comparing the expected file and the output file byte by byte.
>       assert filecmp.cmp(
            f"{pwd}/tests/transformer/testdata/expected/v2/all",
            f"{tmp_path}/v2/all",
        )
E       AssertionError: assert False
E        +  where False = <function cmp at 0x7f3e77209580>('/home/major/git/redhatcloudx/transformer/tests/transformer/testdata/expected/v2/all', '/tmp/pytest-of-major/pytest-0/popen-gw0/test_all0/v2/all')
E        +    where <function cmp at 0x7f3e77209580> = filecmp.cmp

tests/transformer/test_all.py:31: AssertionError

It's not clear what isn't the same between these two files. I can run diff on both files mentioned, but then I need to also pretty print the JSON output to figure out what is missing.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant