Streaming backtest does not stop immediately when margin exceeded

[stastnypremysl]

Bug Report

In given files margin-exceeded-nontermination_bug.py and margin-exceeded-nontermination_bug.out are a strategy script and its output. The strategy exceeds its margin.

Expected Behavior

Strategy immediately terminates after margin exceed is detected.

Actual Behavior

Strategy continues to run for some time.

Comment on severity

If margin exceed happens on the end of the backtesting, there might be a possibility, that it is not terminated correctly with an exception. If not solved, it is needed in my opinion to be sure, that once margin exceeded, the backtest can't finish without an exception raised.

Steps to Reproduce the Problem

1. Download the data
2. Extract the zip file and change extracted catalog path in margin-exceeded-nontermination_bug.py to the catalog
3. ./margin-exceeded-nontermination_bug.py >./margin-exceeded-nontermination_bug.out 2>&1
4. less ./margin-exceeded-nontermination_bug.out

Specifications

• OS platform: Gentoo Linux (native build, amd64)
• Python version: Python 3.12.9 (native build)
• nautilus_trader version: 1.217.0 (revision 37b4a6f; make build-wheel)

Additional notes

• The simulation is running on aggregated trade ticks from Binance data collection
• The instrument information was downloaded using the NT Binance client
• Ignore the 1.217.2 version in the log - no changes in the source codes was made other then change of version

[cjdsellers]

Hi @stastnypremysl

Thanks for the report.

In the provided output I see the following error (related to balance), which stops the backtest:

ValueError(`free` amount was negative)

My understanding is you're concerned that when margin is exceeded on the last iteration of the engine, an exception may not be raised and so this condition would go undetected?

I agree with you that this is not preferable -- the user should always be notified of this in all circumstances.

If this is the expectation, are you able to provide a Python script which clearly reproduces that scenario. This would help us to target a change for the expected behavior efficiently.

[stastnypremysl]

Hi @cjdsellers,

As I wrote, this is only a possibility. This is a bug either way, but with different severity.

I will try to provide an example, where it exceeds margin without a fail or confirm, it always failed in my tests, so we know more and the bug is handled appropriately with correct severity.

[stastnypremysl]

Hi @cjdsellers,

Here is a strategy and its output, in which an exception is not raised, but the margin is exceed. The data are still the same.

[cjdsellers]

Hi @stastnypremysl

Thanks for the report.

The issue here was that the AccountError producing those margin exceeded logs was being handled by the backtest engine (to produce the log), but not reraised to interrupt the streaming of chunks here.

Now fixed from commit 442a95b, the backtest now stops immediately on the first occurrence of margin being exceeded.