Trailing stop market order fill behavior misaligned with market orders for backtests

[stastnypremysl]

Bug Report

There is a backtest in given file.

Expected Behavior

Market orders get filled in one time all the time. (possibly splitted into several trades)

Actual Behavior

Market orders get spitted into several trades, which are filled in different times, after the program is running some time (at most few seconds).

Steps to Reproduce the Problem

1. Download the data
2. Extract the zip file and change extracted catalog path in market-order-time-fragmentation_bug.ipynb to the catalog
3. Run the strategy
4. Find the lines, where given behavior happens (at the end of the log)

Specifications

• OS platform: Gentoo Linux (native build, amd64)
• Python version: Python 3.12.9 (native build)
• nautilus_trader version: 1.217.0 (revision 37b4a6f; make build-wheel)

Additional notes

• The simulation is running on aggregated trade ticks from Binance data collection
• The instrument information was downloaded using the NT Binance client
• The vectorization and other micro-architecture processor instructions and customized optimization might have caused the bug to behave differently on my system
• Ignore the 1.217.2 version in the log - no changes in the source codes was made other then change of version
• This is the second potential memory corruption bug I was speaking on Discord dev-testing channel

[cjdsellers]

Hi @stastnypremysl

Thanks for the report.

This is likely filling the maximum size available based on each trade size. Typically for maximum realism you need to further pre-process your trades data to increase the size available at the top of the book, as Nautilus doesn't make assumptions and will otherwise just use the raw data provided.

I'd also suspect this kind of issue before we make the leap to possible "memory corruption".

Per the discussion on this issue, we can more easily reproduce if the data is readily available in the repo.

[stastnypremysl]

After discussions outside the issue, the data to replicate the given problem (instruments and trade ticks) for given pair was provided.

[cjdsellers]

Hi @stastnypremysl

After further investigation this is a case of not enough size available at the top of the book compared to the quantity of orders being submitted.

The quantity of orders is 10,000 attempting to be matched based on trades data with much smaller size (see small sample below which is involved in the trade with last_qty of 8.2)

2024-12-10T00:00:08.205000000Z [INFO] BACKTESTER-001.MACDStrategy: Filled qty 8.2

[TradeTick(APTUSDT-PERP.BINANCE,11.92460,31.8,BUYER,245141078,1733788808062000000),
 TradeTick(APTUSDT-PERP.BINANCE,11.92500,8.2,BUYER,245141079,1733788808065000000),
 TradeTick(APTUSDT-PERP.BINANCE,11.92560,2.9,SELLER,245141080,1733788808074000000),

This relates to #2564 to improve the execution model with more configs and options. But in the mean time I would suggest either processing your trades data to increase the top of book size to something more realistic, or obtain additional data if your goal is a more accurate simulation.

[stastnypremysl]

Hi, @cjdsellers.

Thanks for the analysis. This really seems as false positive.

I now realized, that this behavior is the most expectable rather then FOK behavior or IOC behavior. But I think, it would be good to have these options in the enhanced model.

Should I put in the ticket?

[cjdsellers]

I don't think FOK or IOC is applicable here, if you want to guarantee an exit for a stop-loss type order then you wouldn't want to use these time in force options.

These execution instructions are available already, and you can test if you like with a more targeted scenario (I suggest grepping for where these are tested already to save time).

The crux of this is that there is insufficient size from raw trades data to simulate book depth properly and thus the execution of the trailing stop order.

For example, here is a quick screenshot of the book on APTUSDT-PERP:

Compare that to the single level size being simulated from the trades data.

[cjdsellers]

After some further investigation we can improve the situation here to make trailing stop market orders behave like market orders for fills.

Problem – Trailing-stop market orders stopped filling after the top-level size was consumed, producing unrealistic partial fills when backtesting with trade-only data.

Fix – After consuming top-level liquidity we now re-price one tick deeper and continue filling until the order quantity is satisfied, mirroring normal market-order behaviour.

Rationale – One-tick slippage is a pragmatic assumption when depth data are unavailable; it yields more realistic fills without over-complicating the model.

Fix/improvement applied from commit b8d89a8.

[stastnypremysl]

Hi, @cjdsellers.

This a great message for me as this was possibly one of the blockers for me. I will recheck my strategy, whether the some of untrustworthy behavior got miss out.

I will also check limit orders and possibly some other types, whether it aligns with the model.