Skip to content

feat: remove feature of filter event with observed generation #902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 2, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 13 additions & 15 deletions docs/documentation/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,17 +137,13 @@ mostly for the cases when there is a long waiting period after a delete operatio
you might want to either schedule a timed event to make sure
`cleanup` is executed again or use event sources to get notified about the state changes of a deleted resource.

## Generation Awareness and Automatic Observed Generation Handling
## Automatic Observed Generation Handling

Having `.observedGeneration` value on the status of the resource is a best practice to indicate the last generation of
the resource reconciled successfully by the controller. This helps the users / administrators to check if the custom
resource was reconciled, but it is used to decide if a reconciliation should happen or not. Filtering events based on
generation is supported by the framework and turned on by default. There are two modes.
resource was reconciled.

### Primary (preferred) Mode

The first and the **preferred** one is to check after a resource event received, if the generation of the resource is
larger than the `.observedGeneration` field on status. In order to have this feature working:
In order to have this feature working:

- the **status class** (not the resource) must implement the
[`ObservedGenerationAware`](https://github.com/java-operator-sdk/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/ObservedGenerationAware.java)
Expand Down Expand Up @@ -189,16 +185,18 @@ public class WebPage extends CustomResource<WebPageSpec, WebPageStatus>
}
```

### The Second (Fallback) Mode
## Generation Awareness and Event Filtering

On an operator startup, the best practice is to reconcile all the resources. Since while operator was down, changes
might have made both to custom resource and dependent resources.

The second, fallback mode is (when the conditions from above are not met to handle the observed generation automatically
in status) to handle generation filtering in memory. Thus, if an event is received, the generation of the received
resource is compared with the resource in the cache.
When the first reconciliation is done successfully, the next reconciliation is triggered if either the dependent
resources are changed or the custom resource `.spec` is changed. If other fields like `.metadata` is changed on the
custom resource, the reconciliation could be skipped. This is supported out of the box, thus the reconciliation by
default is not triggered if the change to the main custom resource does not increase the `.metadata.generation` field.
Note that the increase of `.metada.generation` is handled automatically by Kubernetes.

Note that the **first approach has significant benefits** in the situation when the operator is restarted and there is
no cached resource anymore. In case two this leads to a reconciliation of every resource in all cases,
event if the resource is not changed while the operator was not running. However, in case informers are used
the reconciliation from startup will happen anyway, since events will be propagated by the informer.
To turn on this feature set `generationAwareEventProcessing` to `false` for the `Reconciler`.

## Support for Well Known (non-custom) Kubernetes Resources

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@

/**
* If the custom resource's status implements this interface, the observed generation will be
* automatically handled. The last observed generation will be updated on status. In addition to
* that, controller configuration will be checked if is set to be generation aware. If generation
* aware config is turned off, this interface is ignored.
* automatically handled. The last observed generation will be updated on status.
* <p>
* In order for this automatic handling to work the status object returned by
* {@link CustomResource#getStatus()} should not be null.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
package io.javaoperatorsdk.operator.processing.event.source.controller;

import io.fabric8.kubernetes.api.model.HasMetadata;
import io.fabric8.kubernetes.client.CustomResource;
import io.javaoperatorsdk.operator.api.ObservedGenerationAware;

/**
* Convenience implementations of, and utility methods for, {@link ResourceEventFilter}.
Expand All @@ -25,16 +23,6 @@ public final class ResourceEventFilters {
private static final ResourceEventFilter<HasMetadata> GENERATION_AWARE =
(configuration, oldResource, newResource) -> {
final var generationAware = configuration.isGenerationAware();
if (newResource instanceof CustomResource<?, ?>) {
var newCustomResource = (CustomResource<?, ?>) newResource;
final var status = newCustomResource.getStatus();
if (generationAware && status instanceof ObservedGenerationAware) {
var actualGeneration = newResource.getMetadata().getGeneration();
var observedGeneration = ((ObservedGenerationAware) status)
.getObservedGeneration();
return observedGeneration == null || actualGeneration > observedGeneration;
}
}
return oldResource == null || !generationAware ||
oldResource.getMetadata().getGeneration() < newResource.getMetadata().getGeneration();
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@

import io.fabric8.kubernetes.api.model.HasMetadata;
import io.fabric8.kubernetes.api.model.KubernetesResourceList;
import io.fabric8.kubernetes.api.model.ObjectMeta;
import io.fabric8.kubernetes.client.dsl.MixedOperation;
import io.fabric8.kubernetes.client.dsl.Resource;
import io.javaoperatorsdk.operator.TestUtils;
Expand All @@ -20,7 +19,6 @@
import io.javaoperatorsdk.operator.processing.event.source.controller.ControllerResourceEventSource;
import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceAction;
import io.javaoperatorsdk.operator.processing.event.source.controller.ResourceEventFilter;
import io.javaoperatorsdk.operator.sample.observedgeneration.ObservedGenCustomResource;
import io.javaoperatorsdk.operator.sample.simple.TestCustomResource;

import static org.mockito.Mockito.any;
Expand Down Expand Up @@ -101,27 +99,6 @@ public void eventFilteredByCustomPredicateAndGenerationAware() {
verify(eventHandler, times(1)).handleEvent(any());
}

@Test
public void observedGenerationFiltering() {
var config = new ObservedGenControllerConfig(FINALIZER, true, null);

var eventSource = init(new ObservedGenController(config));

ObservedGenCustomResource cr = new ObservedGenCustomResource();
cr.setMetadata(new ObjectMeta());
cr.getMetadata().setFinalizers(List.of(FINALIZER));
cr.getMetadata().setGeneration(5L);
cr.getStatus().setObservedGeneration(5L);

eventSource.eventReceived(ResourceAction.UPDATED, cr, null);
verify(eventHandler, times(0)).handleEvent(any());

cr.getMetadata().setGeneration(6L);

eventSource.eventReceived(ResourceAction.UPDATED, cr, null);
verify(eventHandler, times(1)).handleEvent(any());
}

@Test
public void eventAlwaysFilteredByCustomPredicate() {
var config = new TestControllerConfig(
Expand All @@ -147,13 +124,6 @@ public TestControllerConfig(String finalizer, boolean generationAware,
super(finalizer, generationAware, eventFilter, TestCustomResource.class);
}
}
private static class ObservedGenControllerConfig
extends ControllerConfig<ObservedGenCustomResource> {
public ObservedGenControllerConfig(String finalizer, boolean generationAware,
ResourceEventFilter<ObservedGenCustomResource> eventFilter) {
super(finalizer, generationAware, eventFilter, ObservedGenCustomResource.class);
}
}

private static class ControllerConfig<T extends HasMetadata> extends
DefaultControllerConfiguration<T> {
Expand Down Expand Up @@ -192,22 +162,4 @@ public MixedOperation<TestCustomResource, KubernetesResourceList<TestCustomResou
}
}

private static class ObservedGenController
extends Controller<ObservedGenCustomResource> {

public ObservedGenController(
ControllerConfiguration<ObservedGenCustomResource> configuration) {
super(null, configuration, null);
}

@Override
public EventSourceManager<ObservedGenCustomResource> getEventSourceManager() {
return mock(EventSourceManager.class);
}

@Override
public MixedOperation<ObservedGenCustomResource, KubernetesResourceList<ObservedGenCustomResource>, Resource<ObservedGenCustomResource>> getCRClient() {
return mock(MixedOperation.class);
}
}
}