.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/plot_rate_bregman.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_auto_examples_plot_rate_bregman.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_plot_rate_bregman.py:


=============================
Nullity of Bregman guarantees
=============================

This example shows that the Bregman divergence is not enough to control
convergence of the iterates: it vanishes as soon as the signs are the same.

.. GENERATED FROM PYTHON SOURCE LINES 9-71



.. image-sg:: /auto_examples/images/sphx_glr_plot_rate_bregman_001.png
   :alt: plot rate bregman
   :srcset: /auto_examples/images/sphx_glr_plot_rate_bregman_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    usetex mode requires TeX.
    /home/circleci/.local/lib/python3.8/site-packages/sklearn/linear_model/_base.py:133: FutureWarning: The default of 'normalize' will be set to False in version 1.2 and deprecated in version 1.4.
    If you wish to scale the data, use Pipeline with a StandardScaler in a preprocessing stage. To reproduce the previous behavior:

    from sklearn.pipeline import make_pipeline

    model = make_pipeline(StandardScaler(with_mean=False), LassoLars())

    If you wish to pass a sample_weight parameter, you need to pass it as a fit parameter to each step of the pipeline as follows:

    kwargs = {s[0] + '__sample_weight': sample_weight for s in model.steps}
    model.fit(X, y, **kwargs)

    Set parameter alpha to: original_alpha * np.sqrt(n_samples). 
      warnings.warn(
    Res norm PD 1.64e-10
    Res norm BP 8.47e-13






|

.. code-block:: default

    import numpy as np
    import matplotlib.pyplot as plt

    from numpy.linalg import norm
    from sklearn.linear_model import LassoLars

    from iterreg.sparse import primal_dual
    from iterreg.utils import make_sparse_data

    from celer.plot_utils import configure_plt
    configure_plt()


    n, d = 30, 70

    snr = 10

    rho = 0.2
    X, y, x_true = make_sparse_data(n, d, rho=rho, snr=snr)

    clf = LassoLars(alpha=1e-20, fit_intercept=False)
    clf.fit(X, y)
    w_bp = clf.coef_

    f_store = 50
    max_iter = 200000
    algo = "Primal-Dual"

    w_pd, theta_pd, all_w_pd = primal_dual(
        X, y, max_iter=max_iter, f_store=f_store, verbose=False)

    subgrad = - X.T @ theta_pd
    assert w_bp[np.abs(subgrad) > 1 - 1e-10].all()
    assert not w_bp[np.abs(subgrad) < 1 - 1e-10].any()
    subgrad2 = np.sign(w_bp)

    bregman = (norm(all_w_pd, ord=1, axis=1) - norm(w_pd, ord=1) -
               (subgrad2[None, :] * (all_w_pd - w_pd)).sum(axis=1))
    feasability = norm(X @ all_w_pd.T - y[:, None], axis=0)
    print("Res norm PD %.2e" % norm(X @ w_pd - y))
    print("Res norm BP %.2e" % norm(X @ w_bp - y))

    plt.close('all')
    figsize = (6.85, 1.75)
    fig, ax = plt.subplots(1, 1, constrained_layout=True, figsize=figsize)

    plot_bregman = True
    n_points = 100
    plt.semilogy(f_store * np.arange(n_points),
                 bregman[:n_points],
                 label=r'$D_{||\cdot||_1}^{- {X}^* \theta^\star}(w_k, w^\star)$')
    plt.semilogy(f_store * np.arange(n_points),
                 norm(all_w_pd - w_bp, axis=1)[:n_points],
                 label=r'$||w_k - w^\star||$')
    plt.semilogy(f_store * np.arange(n_points),
                 feasability[:n_points],
                 label=r'$||{X} w_k - {y}||$')

    plt.xlabel("Iteration $k$")
    plt.legend(loc='upper right')
    ax.set_yticks([1e-1, 1e-6, 1e-11])
    plt.show(block=False)


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  4.936 seconds)


.. _sphx_glr_download_auto_examples_plot_rate_bregman.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: plot_rate_bregman.py <plot_rate_bregman.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: plot_rate_bregman.ipynb <plot_rate_bregman.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_