WIP - FEAT - Quantile Huber & Progressive Smoothing #312
Conversation
floriankozikowski
changed the title
examples/plot_smooth_quantile.py
Outdated
| return np.mean(residuals * (quantile - (residuals < 0))) | ||
|
|
||
|
|
||
| def create_data(n_samples=1000, n_features=10, noise=0.1): |
There was a problem hiding this comment.
avoid this: this is literally just wrapping make_regression
examples/plot_smooth_quantile.py
Outdated
| plt.tight_layout() | ||
| plt.show() | ||
|
|
||
|
|
There was a problem hiding this comment.
no need to wrap in if name == main for example plots
skglm/experimental/quantile_huber.py
Outdated
| res += self._loss_scalar(residual) | ||
| return res / n_samples | ||
|
|
||
| def _loss_scalar(self, residual): |
There was a problem hiding this comment.
loss_sample may be a clearer name
skglm/experimental/quantile_huber.py
Outdated
| grad_j += -X[i, j] * self._grad_scalar(residual) | ||
| return grad_j / n_samples | ||
|
|
||
| def _grad_scalar(self, residual): |
There was a problem hiding this comment.
having gradient_scalar and _grad_scalar is a massive risk of confusion in the future; _grad_per_sample ?
skglm/experimental/quantile_huber.py
Outdated
| return grad_j / n_samples | ||
|
|
||
| def _grad_scalar(self, residual): | ||
| """Calculate gradient for a single residual.""" |
skglm/experimental/quantile_huber.py
Outdated
|
|
||
| def fit(self, X, y): | ||
| """Fit using progressive smoothing: delta_init --> delta_final.""" | ||
| X, y = check_X_y(X, y) |
There was a problem hiding this comment.
no need to check: GeneralizedLinearEstimator will do it
skglm/experimental/quantile_huber.py
Outdated
|
|
||
| for i, delta in enumerate(deltas): | ||
| datafit = QuantileHuber(quantile=self.quantile, delta=delta) | ||
| penalty = L1(alpha=self.alpha) |
There was a problem hiding this comment.
those can be taken out of the for loop
There was a problem hiding this comment.
(initialize datafit, penalty, solver and est outside of the loop; then in the loop only update the delta parameter of GLE.datafit)
skglm/experimental/quantile_huber.py
Outdated
| solver=solver | ||
| ) | ||
|
|
||
| if i > 0: |
There was a problem hiding this comment.
this way you won't need this (if est is fixed outside the loop and uses warm_start=True)
|
|
||
| return self | ||
|
|
||
| def predict(self, X): |
There was a problem hiding this comment.
you can store est as self.est and use self.est.predict to leverage existing code
|
Ok as discussed separately, you need to implement the maths computation to make the solver work with Fista solver and AndersonCD solver; then it should be easy to support the intercept as these solvers rely on |
Context of the PR
This PR implements a smooth quantile regression estimator using a Huberized loss with progressive smoothing. The goal is to provide a faster alternative to scikit-learn's QuantileRegressor while maintaining similar accuracy.
(closes #276 )
(Also it aims to simplify earlier approaches done in PR #306 )
Contributions of the PR
Added QuantileHuber loss in skglm/experimental/quantile_huber.py
Added SmoothQuantileRegressor class in skglm/experimental/smooth_quantile_regressor.py:
Added example in examples/plot_smooth_quantile.py
Checks before merging PR