Skip to content

Update Rust parquet catalog with recent features #2442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 12, 2025
Merged

Conversation

faysou
Copy link
Collaborator

@faysou faysou commented Mar 12, 2025

Pull Request

Update rust parquet catalog with recent features

Added different kinds of adding data to an existing parquet file as well as the ability to consolidate several parquet files into one to match what is already done in python, and timestamp_bound

Type of change

  • New feature (non-breaking change which adds functionality)

How has this change been tested?

Not tested yet

@faysou faysou force-pushed the rustcat branch 5 times, most recently from 83e0a61 to 75156f0 Compare March 12, 2025 17:31
@cjdsellers cjdsellers changed the title Update rust parquet catalog with recent features Update Rust parquet catalog with recent features Mar 12, 2025
@@ -1,5 +1,5 @@
{
"databento_mbo_xnas_itch.csv": "sha256:5481466e7776659f7f093b1ca8b05c7e88b8d586036692089cca5decc7965099",
"databento_mbo_xnas_itch.csv": "sha256:ac66f2775ebbfd71b5ac1e62f47df5d151150b18323eaeb4bcc88b330ee7a456",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's odd this checksum changed, probably a partial download unless Databento data has recently changed.
We're using it for an order book test which isn't fully completed, so not asserting anything.

I'll redownload myself after merge and see if the checksums have changed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it was a partial or corrupted download, as I get the original checksum.

@cjdsellers cjdsellers merged commit 0bbfd5b into develop Mar 12, 2025
12 checks passed
@cjdsellers cjdsellers deleted the rustcat branch March 12, 2025 19:07
@faysou
Copy link
Collaborator Author

faysou commented Mar 12, 2025

Yes I must have commited by accident, something not practical is that make cargo-test changes this file, so often I need to make a rollback of it. Something to fix if possible

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants