diff --git a/doc/source/whatsnew/v0.23.4.txt b/doc/source/whatsnew/v0.23.4.txt
index 5e19ab491647d..a30fbc75f11f8 100644
--- a/doc/source/whatsnew/v0.23.4.txt
+++ b/doc/source/whatsnew/v0.23.4.txt
@@ -27,6 +27,7 @@ Bug Fixes
 **Groupby/Resample/Rolling**
 
 - Bug where calling :func:`DataFrameGroupBy.agg` with a list of functions including ``ohlc`` as the non-initial element would raise a ``ValueError`` (:issue:`21716`)
+- Bug in ``roll_quantile`` caused a memory leak when calling ``.rolling(...).quantile(q)`` with ``q`` in (0,1) (:issue:`21965`)
 -
 
 **Conversion**
diff --git a/pandas/_libs/window.pyx b/pandas/_libs/window.pyx
index 5121d293efcb6..a77433e5d1115 100644
--- a/pandas/_libs/window.pyx
+++ b/pandas/_libs/window.pyx
@@ -1482,6 +1482,8 @@ def roll_quantile(ndarray[float64_t, cast=True] input, int64_t win,
             else:
                 output[i] = NaN
 
+    skiplist_destroy(skiplist)
+
     return output