pv forecast with scrapper and generic pv panel and inverter

[scruysberghs]

@davidusb-geek
For my use case(s) and I guess for many users it would be nice to use scrapper (pvlib + ClearOutside) PV predictions in emhass but instead of having to find an exact (or close enough) matching panel and inverter type in the pvlib database just use a generic panel and inverter size like they do in solcast and solar.forecast and in the home assistant energy dashboard. I see that pvlib supports this notion of a generic system using pvwatts. Is this something you have looked at before? Would you accept a PR that allows sizes in Watt instead of the text string with the exact models?

[davidusb-geek]

Hi, this can be an option. I'm aware of the pvwatts possibility.
Yes of course, we can accept a PR for this. It will be a simpler way to setup. We can keep the advanced setup and propose a simplified setup with your proposition

[scruysberghs]

I left the path to use pvlib from within emhass and have now switched to using https://github.com/rany2/ha-open-meteo-solar-forecast integration from within Home Assistant.
The results are very accurate, you can setup multiple strings/inverters and add them together, and also use the same values in the energy dashboard in Home Assistant. No problems with limited api calls and having to setup solar prediction for HA and emhass separately ...

Template to pass the next pred_horizon number of half hour blocks to the mpc (or day ahead) optimizer:

{% set now = now() %}
              {% set today_data = state_attr('sensor.energy_production_today', 'watts') %}
              {% set tomorrow_data = state_attr('sensor.energy_production_tomorrow', 'watts') %}
              {% set combined_data = 
                  (today_data | dictsort | selectattr('0', '>=', now.isoformat()) | list) 
                  +
                  (tomorrow_data | dictsort | list)
              %}
              {% set sorted_data = combined_data | sort(attribute='0') %}
              {% set next_24h = sorted_data | selectattr('0', '<', (now + timedelta(hours=24)).isoformat()) | list %}
              "pv_power_forecast": [{{states('sensor.sec_pac_solar') | int(0)}},{% for i in range(2, pred_horizon*2, 2) %}{{ ((next_24h[i][1] + next_24h[i + 1][1]) / 2) | round(0) }}{{ ", " if not loop.last else "" }}{% endfor %}],

[davidusb-geek]

Very nice solution using that custom integration

[scruysberghs]

is see openmeteo has made it into emhass... great.
I do feel that it is a little wastefull to pull solar predictions in emhass while almost everyone is also pulling in the same solar predictions into Home Assistant directly for the energy dashboard. I always try to avoid pulling the same remote datasource twice. I guess passing solar predictions from HA as runtime parameters might be too far fetched for most users but maybe a weather_forecast_method called "home assistant" that uses data from the sensor(s) created by the open meteo solar integration in HA that are set in the energy dashboard?
Or the other way around, and have emhass pull the openweather api with the inverters and strings that are defined in the config, but post the attibutes in sesnor.p_pv_forecast in a way that the sensor can also be used in the energy dashboard in Home Assistant?

[scruysberghs]

would also be nice to just set inverter size(s) and panel size(s) instead of finding a somewhat matching inverter or panel in an long but outdated list ... Could be me but I have not come across one inverter used here in Belgium that is in the pvlib list of inverters.

I mean, parameters like the openmeteo solar forecast integration in HA allows you to set

[davidusb-geek]

Yes as I said before we could implement a simplified method for PV modeling maybe it is enough for our needs, contributions are welcomed. However, this being said, if one is able to find the correct models for panels and inverter (in my case I did) on the PVLib database then that will be a more precise modeling of your PV install, the best you could get, proven and functional. But yes, sometimes it can be hard to find the correct models in the database, the dedicated webapp was a way to ease this process.

[scruysberghs]

thanks David. Just tried another query in the database (https://emhass-pvlib-database.streamlit.app/) for the Solaredge SE8K-RWS48BEN4 (8kW, hybrid), but nothing...

For now, people who want to use generic inverter(s) and panels, you can define multiple strings and inverters bij adding multiple "Services" in the HA addon like this:

To pass the resulting pv_power_forecast stringas runtime parameter (I use 24 hr pred horizon and 30 min blocks now) you can use this template:

              {% set now = now() %}
              {# Forecast 1 #}
              {% set today_data_1 = state_attr('sensor.energy_production_today', 'watts') or {} %}
              {% set tomorrow_data_1 = state_attr('sensor.energy_production_tomorrow', 'watts') or {} %}
              {% set combined_data_1 = (today_data_1 | dictsort | selectattr('0', '>=', now.isoformat()) | list) + (tomorrow_data_1 | dictsort | list) %}
              {% set next_24h_1 = combined_data_1 | selectattr('0', '<', (now + timedelta(hours=24)).isoformat()) | list %}

              {# Forecast 2 #}
              {% set today_data_2 = state_attr('sensor.energy_production_today_1, 'watts') or {} %}
              {% set tomorrow_data_2 = state_attr('sensor.energy_production_tomorrow_1', 'watts') or {} %}
              {% set combined_data_2 = (today_data_2 | dictsort | selectattr('0', '>=', now.isoformat()) | list) + (tomorrow_data_2 | dictsort | list) %}
              {% set next_24h_2 = combined_data_2 | selectattr('0', '<', (now + timedelta(hours=24)).isoformat()) | list %}

              {# Forecast 3 #}
              {% set today_data_3 = state_attr('sensor.energy_production_today_2', 'watts') or {} %}
              {% set tomorrow_data_3 = state_attr('sensor.energy_production_tomorrow_2', 'watts') or {} %}
              {% set combined_data_3 = (today_data_3 | dictsort | selectattr('0', '>=', now.isoformat()) | list) + (tomorrow_data_3 | dictsort | list) %}
              {% set next_24h_3 = combined_data_3 | selectattr('0', '<', (now + timedelta(hours=24)).isoformat()) | list %}

              {# Build final combined forecast #}
              "pv_power_forecast": [{{ states('sensor<my realtime solar sensor>') | int(0) }},{% for i in range(2, pred_horizon * 2, 2) -%}
                {{ ((next_24h_1[i][1] + next_24h_2[i][1] + next_24h_3[i][1] +
                    next_24h_1[i + 1][1] + next_24h_2[i + 1][1] + next_24h_3[i + 1][1]
                    ) / 2) | round(0) }}{{ ", " if not loop.last else "" }}{%- endfor %}],

You can also add all 3 (or more) predictions in HA energy dashboard to get the dotted line or lines.

[scruysberghs]

But would be nice to able to use the autocorrecting PV prediction machine learning feature I saw mentioned in the future....

[davidusb-geek]

That is cooking, and will be available regardless your forecast or data passing method. Using the templates you show should be fine.