diff --git a/opensearch/file-cache.md b/opensearch/file-cache.md new file mode 100644 index 00000000000..63cdb83d48e --- /dev/null +++ b/opensearch/file-cache.md @@ -0,0 +1,84 @@ +# File Cache + +The File Cache is a component in OpenSearch that manages caching of index files. This documentation covers the recent changes implemented in the File Cache system, specifically the addition of a method to retrieve the reference count of entries in the cache. + +## GetRef Method + +A new method `getRef` has been added to the `FileCache` class. This method allows for retrieving the reference count of a specific entry in the file cache. + +### Method Signature + +```java +public Integer getRef(Path key) +``` + +### Parameters + +- `key`: A `Path` object representing the key of the cache entry. + +### Return Value + +- Returns an `Integer` representing the reference count of the specified cache entry. +- If the entry does not exist in the cache, the method returns `null`. + +### Usage + +The `getRef` method can be used to check the current reference count of a file in the cache. This is particularly useful for debugging and monitoring the cache state. + +Example usage: + +```java +FileCache fileCache = // ... initialize file cache +Path filePath = // ... path to the file +Integer refCount = fileCache.getRef(filePath); +if (refCount != null) { + System.out.println("Reference count for " + filePath + ": " + refCount); +} else { + System.out.println("File " + filePath + " is not in the cache."); +} +``` + +## Implementation Details + +The `getRef` method is implemented in the `FileCache` class and delegates to the underlying cache implementation. The actual retrieval of the reference count is performed by the `LRUCache` class, which manages the cache entries. + +In the `LRUCache` class, the `getRef` method is implemented as follows: + +```java +@Override +public Integer getRef(K key) { + Objects.requireNonNull(key); + lock.lock(); + try { + Node node = data.get(key); + if (node != null) { + return node.refCount; + } + return null; + } finally { + lock.unlock(); + } +} +``` + +This implementation ensures thread-safety by using a lock when accessing the cache data. + +## Use Cases + +The ability to retrieve reference counts for cache entries can be beneficial in several scenarios: + +1. Debugging: Helps in identifying potential reference count issues or leaks. +2. Cache management: Allows for more informed decisions about cache eviction and optimization. +3. Monitoring: Enables better insights into cache usage patterns and performance. + +## Note on Concurrency + +As with other operations on the `FileCache`, the `getRef` method is designed to be thread-safe. However, due to the concurrent nature of the cache, the reference count returned by this method should be considered a snapshot that may change immediately after the method returns. + +## Related Classes + +- `FileCache`: The main class that exposes the `getRef` method. +- `LRUCache`: The underlying cache implementation that manages reference counts. +- `SegmentedCache`: A wrapper around multiple `LRUCache` instances for improved concurrency. + +These changes enhance the observability and management capabilities of the File Cache system in OpenSearch. \ No newline at end of file diff --git a/release-notes/opensearch-documentation-release-notes-2.19.0.md b/release-notes/opensearch-documentation-release-notes-2.19.0.md index f71087df847..2befb33581f 100644 --- a/release-notes/opensearch-documentation-release-notes-2.19.0.md +++ b/release-notes/opensearch-documentation-release-notes-2.19.0.md @@ -1,36 +1,94 @@ -# OpenSearch Documentation Website 2.19.0 Release Notes - -The OpenSearch 2.19.0 documentation includes the following additions and updates. - -## New documentation for 2.19.0 - -- Adds star-tree search changes related to new aggregations supported [#9163](https://github.com/opensearch-project/documentation-website/pull/9163) -- Adds QueryInsightsDashboard [#9157](https://github.com/opensearch-project/documentation-website/pull/9157) -- Add Convert Index to remote documentation [#9156](https://github.com/opensearch-project/documentation-website/pull/9156) -- Adds details for using index_thread_qty for Lucene library [#9152](https://github.com/opensearch-project/documentation-website/pull/9152) -- Update threshold for 2.19 [#9151](https://github.com/opensearch-project/documentation-website/pull/9151) -- Add avx512_spr documentation [#9148](https://github.com/opensearch-project/documentation-website/pull/9148) -- Add documentation for plugin as a service [#9144](https://github.com/opensearch-project/documentation-website/pull/9144) -- Add AD flatten result index feature [#9140](https://github.com/opensearch-project/documentation-website/pull/9140) -- Add documentation for `wait_for_completion_timeout` Parameter [#9138](https://github.com/opensearch-project/documentation-website/pull/9138) -- Add feature direction to AD docs [#9137](https://github.com/opensearch-project/documentation-website/pull/9137) -- Add verbose_pipeline section in using-search-pipeline [#9130](https://github.com/opensearch-project/documentation-website/pull/9130) -- Query Insights 2.19 documentation [#9120](https://github.com/opensearch-project/documentation-website/pull/9120) -- Add template query [#9119](https://github.com/opensearch-project/documentation-website/pull/9119) -- Add RRF documentation for hybrid search [#9117](https://github.com/opensearch-project/documentation-website/pull/9117) -- Add Pagination in hybrid query [#9109](https://github.com/opensearch-project/documentation-website/pull/9109) -- Mark nmslib references for vector search as deprecated [#9107](https://github.com/opensearch-project/documentation-website/pull/9107) -- Add in-place SSL certs hot reload documentation [#9103](https://github.com/opensearch-project/documentation-website/pull/9103) -- Add binary Lucene vector updates for 2.19 [#9102](https://github.com/opensearch-project/documentation-website/pull/9102) -- Add OpenSearch Flow OSD plugin [#9101](https://github.com/opensearch-project/documentation-website/pull/9101) -- Update hot reload documentation to show how DN validation can be skipped [#9079](https://github.com/opensearch-project/documentation-website/pull/9079) -- Update k-NN Cosine formula [#9078](https://github.com/opensearch-project/documentation-website/pull/9078) -= Update Ada Grad as the default optimiser. [#9061](https://github.com/opensearch-project/documentation-website/pull/9061) -- Add documentation about explain in hybrid query and hybrid_score_explanation processor [#9053](https://github.com/opensearch-project/documentation-website/pull/9053) -- Add Query Insights local index delete after documentation [#9052](https://github.com/opensearch-project/documentation-website/pull/9052) -- Add support for Bedrock Rerank API #9027 [#9029](https://github.com/opensearch-project/documentation-website/pull/9029) -- Add validation requirement for message fields [#9000](https://github.com/opensearch-project/documentation-website/pull/9000) -- Add documentation for workspace privacy [#8994](https://github.com/opensearch-project/documentation-website/pull/8994) -- Add documentation for pruning neural sparse vectors [#8984](https://github.com/opensearch-project/documentation-website/pull/8984) -- Add document the usage of update document API with ingest pipeline [#8874](https://github.com/opensearch-project/documentation-website/pull/8874) -- Update innerHits of nested k-NN fields [#8822](https://github.com/opensearch-project/documentation-website/pull/8822) +# OpenSearch Documentation Release Notes 2.19.0 + +## New Features + +### File Cache Reference Count Retrieval + +A new method has been exposed to retrieve the reference count of an entry in the FileCache. This feature allows for better management and monitoring of file cache entries. + +#### Changes in FileCache.java + +The `FileCache` class has been updated with a new method: + +```java +@Override +public Integer getRef(Path key) { + return theCache.getRef(key); +} +``` + +This method allows retrieving the reference count for a given key in the file cache. + +#### Changes in LRUCache.java + +The `LRUCache` class now implements the `getRef` method: + +```java +@Override +public Integer getRef(K key) { + Objects.requireNonNull(key); + lock.lock(); + try { + Node node = data.get(key); + if (node != null) { + return node.refCount; + } + return null; + } finally { + lock.unlock(); + } +} +``` + +This implementation provides thread-safe access to the reference count of a cache entry. + +#### Changes in RefCountedCache.java + +The `RefCountedCache` interface has been updated to include the `getRef` method: + +```java +/** + * get the reference count for key {@code key}. + */ +Integer getRef(K key); +``` + +#### Changes in SegmentedCache.java + +The `SegmentedCache` class now implements the `getRef` method: + +```java +@Override +public Integer getRef(K key) { + if (key == null) throw new NullPointerException(); + return segmentFor(key).getRef(key); +} +``` + +This implementation delegates the reference count retrieval to the appropriate cache segment. + +## Impact and Usage + +This new feature allows developers and system administrators to: + +1. Monitor the usage of specific entries in the file cache +2. Implement more sophisticated cache management strategies +3. Debug potential issues related to file cache reference counting + +To use this new functionality, call the `getRef` method on a `FileCache` instance with the desired key: + +```java +Integer refCount = fileCache.getRef(path); +``` + +This will return the current reference count for the specified path in the file cache, or `null` if the entry doesn't exist. + +## Compatibility + +This change is backwards compatible and does not affect existing functionality. It only adds a new method to retrieve information that was previously not accessible. + +## Testing + +The pull request includes additional test cases to verify the correct implementation of the new `getRef` method across various cache implementations. + +For more detailed information, please refer to the [pull request #18259](https://github.com/opensearch-project/OpenSearch/pull/18259) in the OpenSearch repository.