feat(flagd): add http connector for In-process resolver

pom.xml

@@ -40,6 +40,7 @@
         <module>providers/configcat</module>
         <module>providers/statsig</module>
         <module>providers/multiprovider</module>
+        <module>tools/flagd-http-connector</module>
     </modules>
 
     <scm>

tools/flagd-http-connector/CHANGELOG.md

@@ -0,0 +1 @@
+# Changelog

tools/flagd-http-connector/README.md

@@ -0,0 +1,186 @@
+# Http Connector
+
+## Introduction
+Http Connector is a tool for [flagd](https://github.com/open-feature/flagd) in-process resolver.
+
+This mode performs flag evaluations locally (in-process).  
+Flag configurations for evaluation are obtained via Http.
+
+## Http Connector functionality
+
+HttpConnector is responsible for polling data from a specified URL at regular intervals.  
+It is leveraging Http cache mechanism with 'ETag' header, then when receiving 304 Not Modified response,
+reducing traffic, reducing rate limits effects and changes updates. Can be enabled via useHttpCache option.  
+The implementation is using Java HttpClient.
+
+## Use cases and benefits
+* flagd installation is not required, the Http Connector works independently.  
+  Minimizing infrastructure and DevOps overhead - no extra containers required.
+* Low latency by fetching data directly in-process.
+* Decreased external network traffic from the HTTP source, even without a standalone flagd container or proxy, 
+  when using polling cache.
+* Can serve as an additional provider for fallback or internal backup scenarios using a multi-provider setup.
+
+### What happens if the Http source is down during application startup?
+
+Http Connector supports optional resilient fail-safe initialization using a cache.
+If the initial fetch fails due to source unavailability, it can load the initial payload from the cache instead of
+falling back to default values.
+This ensures smoother startup behavior until the source becomes available again. To be effective, the TTL of the
+fallback cache should be longer than the expected duration of the source downtime during initialization.
+
+### Polling cache
+The polling cache is used to store the payload fetched from the URL.  
+Used when usePollingCache is configured as true.
+A key advantage of this cache is that it enables a single microservice within a cluster to handle the polling of a 
+URL, effectively acting as a flagd/proxy while all other services leverage the shared cache. 
+This approach optimizes resource usage by preventing redundant polling across services.
+
+### Sample flows demonstrating the architecture
+
+#### Basic Simple Configuration
+
+This example demonstrates a simple flow using:
+- GitHub as the source for flag payload.
+
+```mermaid
+sequenceDiagram
+    participant service
+    participant Github
+
+    service->>Github: fetch
+    Github->>service: payload
+    Note right of service: polling interval passed
+    service->>Github: fetch
+    Github->>service: payload
+```
+
+#### A More Scalable Configuration Utilizing Fail-Safe and Polling Caching Mechanisms
+
+This configuration aim to reduce network requests to the source URL, to improve performance and to improve the
+application's resilience to source downtime.
+
+This example demonstrates a micro-services architectural flow using:
+- GitHub as the source for flag payload.
+- Redis serving as both the fail-safe initialization cache and the polling cache.
+
+Example initialization flow during GitHub downtime,
+demonstrates how the application continues to access flag values from the cache even when GitHub is unavailable.
+In this setup, multiple microservices share the same cache, with only one service responsible for polling the source
+URL.
+
+```mermaid
+sequenceDiagram
+    box Cluster
+    participant micro-service-1
+    participant micro-service-2
+    participant micro-service-3
+    participant Redis
+    end
+    participant Github
+
+    break source downtime
+    micro-service-1->>Github: initialize
+    Github->>micro-service-1: failure
+    end
+    micro-service-1->>Redis: fetch
+    Redis->>micro-service-1: failsafe payload
+    Note right of micro-service-1: polling interval passed
+    micro-service-1->>Github: fetch
+    Github->>micro-service-1: payload
+    micro-service-2->>Redis: fetch
+    Redis->>micro-service-2: payload
+    micro-service-3->>Redis: fetch
+    Redis->>micro-service-3: payload
+
+```
+
+## Usage
+
+### Installation
+<!-- x-release-please-start-version -->
+```xml
+<dependency>
+  <groupId>dev.openfeature.contrib.tools</groupId>
+  <artifactId>flagd-http-connector</artifactId>
+  <version>0.0.1</version>
+</dependency>
+```
+<!-- x-release-please-end-version -->
+
+### Usage example
+
+```java
+HttpConnectorOptions httpConnectorOptions = HttpConnectorOptions.builder()
+    .url("http://example.com/flags")
+    .build();
+HttpConnector connector = HttpConnector.builder()
+    .httpConnectorOptions(httpConnectorOptions)
+    .build();
+
+FlagdOptions options =
+    FlagdOptions.builder()
+        .resolverType(Config.Resolver.IN_PROCESS)
+        .customConnector(connector)
+        .build();
+
+FlagdProvider flagdProvider = new FlagdProvider(options);
+```
+
+#### HttpConnector using fail-safe cache and polling cache
+
+```java
+PayloadCache payloadCache = new PayloadCache() {
+
+    @Override
+    public void put(String key, String payload) {
+        // implement put in cache
+    }
+
+    @Override
+    public void put(String key, String payload, int ttlSeconds) {
+        // implement put in cache with TTL
+    }
+
+    @Override
+    public String get(String key) {
+        // implement get from cache and return
+        }
+};
+
+HttpConnectorOptions httpConnectorOptions = HttpConnectorOptions.builder()
+    .url(testUrl)
+    .useHttpCache(true)
+    .payloadCache(payloadCache)
+    .payloadCacheOptions(PayloadCacheOptions.builder().build())
+    .useFailsafeCache(true)
+    .pollIntervalSeconds(10)
+    .usePollingCache(true)
+    .build();
+
+HttpConnector connector = HttpConnector.builder()
+    .httpConnectorOptions(httpConnectorOptions)
+    .build();
+```
+
+### Configuration
+The Http Connector can be configured using the following properties in the `HttpConnectorOptions` class.:
+
+| Property Name                             | Type                | Description                                                                                                                                                                                                                                                                                                                                                                         |
+|-------------------------------------------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| url                                       | String              | The URL to poll for updates. This is a required field.                                                                                                                                                                                                                                                                                                                              |
+| pollIntervalSeconds                       | Integer             | The interval in seconds at which the connector will poll the URL for updates. Default is 60 seconds.                                                                                                                                                                                                                                                                                |
+| connectTimeoutSeconds                     | Integer             | The timeout in seconds for establishing a connection to the URL. Default is 10 seconds.                                                                                                                                                                                                                                                                                             |
+| requestTimeoutSeconds                     | Integer             | The timeout in seconds for the request to complete. Default is 10 seconds.                                                                                                                                                                                                                                                                                                          |
+| linkedBlockingQueueCapacity               | Integer             | The capacity of the linked blocking queue used for processing requests. Default is 100.                                                                                                                                                                                                                                                                                             |
+| scheduledThreadPoolSize                   | Integer             | The size of the scheduled thread pool used for processing requests. Default is 2.                                                                                                                                                                                                                                                                                                   |
+| headers                                   | Map<String, String> | A map of headers to be included in the request. Default is an empty map.                                                                                                                                                                                                                                                                                                            |
+| httpClientExecutor                        | ExecutorService     | The executor service used for making HTTP requests. Default is a fixed thread pool with 1 thread.                                                                                                                                                                                                                                                                                   |
+| proxyHost                                 | String              | The host of the proxy server to use for requests. Default is null.                                                                                                                                                                                                                                                                                                                  |
+| proxyPort                                 | Integer             | The port of the proxy server to use for requests. Default is null.                                                                                                                                                                                                                                                                                                                  |
+| payloadCacheOptions                       | PayloadCacheOptions | Options for configuring the payload cache. Default is null.                                                                                                                                                                                                                                                                                                                         |
+| payloadCache                              | PayloadCache        | The payload cache to use for caching responses. Default is null.                                                                                                                                                                                                                                                                                                                    |
+| useHttpCache                              | Boolean             | Whether to use HTTP caching for the requests. Default is false.                                                                                                                                                                                                                                                                                                                     |
+| useFailsafeCache                          | Boolean             | Whether to use a failsafe cache for initialization. Default is false.                                                                                                                                                                                                                                                                                                               |
+| usePollingCache                           | Boolean             | Whether to use a polling cache for initialization. Default is false.                                                                                                                                                                                                                                                                                                                |
+| PayloadCacheOptions.updateIntervalSeconds | Integer             | The interval, in seconds, at which the cache is updated. By default, this is set to 30 minutes. The goal is to avoid overloading fallback cache writes, since the cache serves only as a fallback mechanism. Typically, this value can be tuned to be shorter than the cache's TTL, balancing the need to minimize unnecessary updates while still handling edge cases effectively. |

tools/flagd-http-connector/pom.xml

@@ -0,0 +1,49 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>dev.openfeature.contrib</groupId>
+        <artifactId>parent</artifactId>
+        <version>[0.2,)</version>
+        <relativePath>../../pom.xml</relativePath>
+    </parent>
+    <groupId>dev.openfeature.contrib.tools</groupId>
+    <artifactId>flagdhttpconnector</artifactId>
+    <version>0.0.1</version> <!--x-release-please-version -->
+
+    <name>flagd-http-connector</name>
+    <description>Flagd Http Connector</description>
+    <url>https://openfeature.dev</url>
+
+    <developers>
+        <developer>
+            <id>liran2000</id>
+            <name>Liran Mendelovich</name>
+            <organization>OpenFeature</organization>
+            <url>https://openfeature.dev/</url>
+        </developer>
+    </developers>
+
+    <dependencies>
+        <dependency>
+            <groupId>dev.openfeature.contrib.providers</groupId>
+            <artifactId>flagd</artifactId>
+            <version>0.11.8</version>
+        </dependency>
+
+        <dependency>
+            <groupId>org.apache.commons</groupId>
+            <artifactId>commons-lang3</artifactId>
+            <version>3.17.0</version>
+        </dependency>
+
+        <dependency>
+            <groupId>com.google.code.findbugs</groupId>
+            <artifactId>annotations</artifactId>
+            <version>3.0.1</version>
+            <scope>test</scope>
+        </dependency>
+
+    </dependencies>
+</project>

tools/flagd-http-connector/src/main/java/dev/openfeature/contrib/tools/flagd/resolver/process/storage/connector/sync/http/FailSafeCache.java

@@ -0,0 +1,84 @@
+package dev.openfeature.contrib.tools.flagd.resolver.process.storage.connector.sync.http;
+
+import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
+import lombok.Builder;
+import lombok.extern.slf4j.Slf4j;
+
+/**
+ * A wrapper class for managing a payload cache with a specified update interval.
+ * This class ensures that the cache is only updated if the specified time interval
+ * has passed since the last update. It logs debug messages when updates are skipped
+ * and error messages if the update process fails.
+ * Not thread-safe.
+ *
+ * <p>Usage involves creating an instance with {@link PayloadCacheOptions} to set
+ * the update interval, and then using {@link #updatePayloadIfNeeded(String)} to
+ * conditionally update the cache and {@link #get()} to retrieve the cached payload.</p>
+ */
+@SuppressFBWarnings(
+        value = {"EI_EXPOSE_REP2", "CT_CONSTRUCTOR_THROW"},
+        justification = "builder validations")
+@Slf4j
+public class FailSafeCache {
+    public static final String FAILSAFE_PAYLOAD_CACHE_KEY = FailSafeCache.class.getSimpleName() + ".failsafe-payload";
+    private long lastUpdateTimeMs;
+    private long updateIntervalMs;
+    private PayloadCache payloadCache;
+
+    /**
+     * Constructor for FailSafeCache.
+     *
+     * @param payloadCache the payload cache to be used
+     * @param payloadCacheOptions the options for configuring the cache
+     */
+    @Builder
+    public FailSafeCache(PayloadCache payloadCache, PayloadCacheOptions payloadCacheOptions) {
+        validate(payloadCacheOptions);
+        this.updateIntervalMs = payloadCacheOptions.getUpdateIntervalSeconds() * 1000L;
+        this.payloadCache = payloadCache;
+    }
+
+    private static void validate(PayloadCacheOptions payloadCacheOptions) {
+        if (payloadCacheOptions.getUpdateIntervalSeconds() < 1) {
+            throw new IllegalArgumentException("pollIntervalSeconds must be larger than 0");
+        }
+    }
+
+    /**
+     * Updates the payload in the cache if the specified update interval has passed.
+     *
+     * @param payload the payload to be cached
+     */
+    public void updatePayloadIfNeeded(String payload) {
+        if ((getCurrentTimeMillis() - lastUpdateTimeMs) < updateIntervalMs) {
+            log.debug("not updating payload, updateIntervalMs not reached");
+            return;
+        }
+
+        try {
+            log.debug("updating payload");
+            payloadCache.put(FAILSAFE_PAYLOAD_CACHE_KEY, payload);
+            lastUpdateTimeMs = getCurrentTimeMillis();
+        } catch (Exception e) {
+            log.error("failed updating cache", e);
+        }
+    }
+
+    protected long getCurrentTimeMillis() {
+        return System.currentTimeMillis();
+    }
+
+    /**
+     * Retrieves the cached payload.
+     *
+     * @return the cached payload
+     */
+    public String get() {
+        try {
+            return payloadCache.get(FAILSAFE_PAYLOAD_CACHE_KEY);
+        } catch (Exception e) {
+            log.error("failed getting from cache", e);
+            return null;
+        }
+    }
+}