Skip to content
This repository was archived by the owner on May 17, 2024. It is now read-only.
This repository was archived by the owner on May 17, 2024. It is now read-only.

diff_tables fails to compare StringType with Native_UUID #245

Closed
@bcodell

Description

@bcodell

Description:

When using the diff_tables method, where table 1 is a Snowflake table, table 2 is a Postgres table, the table 1 key is a String_UUID, and the table 2 key is a Native_UUID, the following error is thrown:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/x/.local/share/virtualenvs/data-env-XwH2fLAj/lib/python3.10/site-packages/data_diff/diff_tables.py", line 163, in diff_tables
    raise error
  File "/Users/x/.local/share/virtualenvs/data-env-XwH2fLAj/lib/python3.10/site-packages/data_diff/diff_tables.py", line 103, in diff_tables
    self._validate_and_adjust_columns(table1, table2)
  File "/Users/x/.local/share/virtualenvs/data-env-XwH2fLAj/lib/python3.10/site-packages/data_diff/diff_tables.py", line 210, in _validate_and_adjust_columns
    raise TypeError(f"Incompatible types for column '{c1}':  {col1} <-> {col2}")
TypeError: Incompatible types for column 'ID':  <data_diff.databases.database_types.String_UUID object at 0x14f123e50> <-> <data_diff.databases.database_types.Native_UUID object at 0x14f120700>

Code used:

pg_table = connect_to_table(pg_connect_str, "my_table", "id")
sf_table = connect_to_table(sf_connect_str, f"DST_SCHEMA.MY_TABLE", "ID")
for different_row in diff_tables(sf_table, pg_table):
    plus_or_minus, columns = different_row
    print(plus_or_minus, columns)

Notes:

  • The error is not reproduced when swapping the table_1 and table_2 arguments to data_diff (i.e. data_diff(pg_table, sf_table) runs as expected)

Environment:

  • MacBook Pro 2021 with M1 Max chip
  • MacOS 12.1 Monterey
  • Python version 3.10.1
  • data_diff version 0.2.6

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions