Skip to content

OpenSearch demo plugin#

This is a demo plugin that integrates token-based access control into OpenSearch. Specifically it is focused on filtering search results obtained in response to search queries sent to any of the endpoints listed here. Filtering takes place based on the Cedarling policy provided in the plugin settings.

Requisites#

  • OpenSearch 3.6.0
  • Jans Server 1.16.0 or higher
  • A browser with tarp extension installed
  • Basic Cedar and OpenSearch knowledge
  • Java 21 and git for development

Notes:

  • All commands given in this document were tested using Ubuntu 22. Accommodate to your specific OS
  • The OpenSearch installation used to test here was single node and packaged-based. Installers can be found here for several OSes. Keep the admin password at hand

Create an .netrc file#

To avoid typing the OpenSearch password in curl commands over and over, create a ~/.netrc file. Here is how it might look:

machine localhost login admin password secret

Run the health check#

With curl, issue a request to the cluster health endpoint to check status. By default all API requests are served through port 9200, e.g. https://localhost:9200. It may take some administrative work to properly issue requests from the development machine, however sending curl requests directly from the server where OpenSearch resides is OK for testing purposes.

In this document, occurrences of https://oshost will refer to the root URL where OpenSearch HTTP API can be reached.

Create and test a cedar policy#

Here, the intention is to create a policy that looks like:

@id("alumni_restricted_access")
permit(
  principal,
  action in Jans::Action::"Search",
  resource is Jans::student
)
when {
  resource.grad_year < 2026 ||
  (
    context has tokens.jans_userinfo_token &&
    context.tokens.jans_userinfo_token.hasTag("role") &&
    context.tokens.jans_userinfo_token.getTag("role").contains("AdmissionsCounselor")
  )
};

with the student entity type in the schema:

{
    "shape": {
        "type": "Record",
        "attributes": {
            "name": {
                "type": "String"
            },
            "grad_year": {
                "type": "Long"
            }
        }
    }
}

For this, follow the steps found here as a guide. Note it is highly recommended to use Agama lab's policy designer in this case as well as Tarp for quickly testing the policy.

For the short of time, there is a readily available policy store here.

Notes:

  • Ensure student is associated to the Search action
  • In Tarp, verify the issued UserInfo tokens have the role claim. One way to do so is adding the role database attribute to the profile OpenId scope in the server settings. In this case the registered client should have such scope associated as well as authentication requests

Plugin deployment#

In the development machine, clone this repository (a shallow clone is recommended). cd to the demos/opensearch-cedarling and run ./gradlew assemble -Dopensearch.version=3.6.0. Ensure JAVA_HOME environment points to a Java 21 installation, e.g. export JAVA_HOME=/path/to/corretto-21.

cd to build/distributions. And run:

  • unzip cedarling.zip 'cedarling-java*'
  • zip -q -d cedarling-java-2.0.0.jar 'com/sun/jna/*'
  • zip -u cedarling.zip cedarling-java-2.0.0.jar

Transfer the file cedarling.zip to the OpenSearch server. In a terminal run the below:

  • /usr/share/opensearch/bin/opensearch-plugin remove cedarling (not required for first time deployment)
  • /usr/share/opensearch/bin/opensearch-plugin install file:///path/to/cedarling.zip
  • systemctl restart opensearch.service

Verify the plugin was effectively deployed by running curl -n https://oshost/_cat/plugins. There should be an entry for cedarling in the table.

Plugin configuration#

OpenSearch provides endpoints for handling configuration settings of plugins as well as Java classes for reading those. However when it comes to large, complex, and nested JSON hierarchies, like the settings this plugin may require, OpenSearch facilities are not convenient. For this purpose, an additional endpoint was implemented in order to retrieve and supply configuration settings. This allows to pass complex JSON objects without hassle. Under the hood everything is converted into a big string and stored as a single setting in Opensearch.

Check the file settings.json and fill the value corresponding to the policy store URI.

With a complete settings file, transfer it to the server and call the endpoint /_plugins/cedarling/settings:

curl -n -H 'Content-Type: application/json' -d @settings.json -X PUT https://oshost/_plugins/cedarling/settings

The response will contain a boolean value indicating the operation success. The current settings can be retrieved issuing a GET to the same endpoint. To check the actual settings OpenSearch stores, use a request like:

curl -n https://oshost/_cluster/settings?pretty

This plugin settings are dynamic and persistent which means they can be altered any time and survive server restarts. Check this page for more information on these concepts.

Setup testing data#

In the example policy, the student entity type is part of the schema. It is expected all resources referenced by policies exist as OpenSearch indices in equivalence. For this, some "students" should be added:

curl -n -H 'Content-Type: application/json' --data-binary @records.txt https://oshost/student/_bulk

Here is a sample records.txt file. Insert more similar documents varying the grad_year. Learn more about Opensearch bulk requests here.

Issue a request to retrieve the documents added so far:

curl -n -H 'Content-Type: application/json' -d @query.json https://oshost/student/_search?pretty

query.json contains a search request that matches all documents in the index.

Note that in real-world scenarios, indices already exist and policies are built in conformance afterward. Every resource to add in the schema should resemble existing indices structures. More specifically, resources should at least contain the attributes which are needed for policy evaluation.

Setup a search pipeline#

This plugin implements a search response processor which must be "attached" to a search pipeline. Transfer the file pipeline.json to the OpenSearch server and run:

curl -n -H 'Content-Type: application/json' -d @pipeline.json -X PUT https://oshost/_search/pipeline/cedarling_search?pretty

This action needs to be performed only once regardless of how many plugin redeployments take place.

With this pipeline, search results may now be filtered as per defined Cedarling policies. See the next section.

Test#

Open Tarp. If this is the first time you use it, add a client there beforehand. Then, in the "Authentication Flow" tab, trigger an authentication flow with the following:

  • Acr: basic
  • Scope: openid and profile
  • Check "Display tokens"

After logging in, copy the UserInfo token and paste it in the corresponding section inside file query_ext.json. This is an "extended" search query the plugin will have access to so the Cedarling engine can be supplied with tokens and contextual data to make decisions.

Then, run:

curl -n -H 'Content-Type: application/json' -d @query_ext.json 'https://oshost/student/_search?pretty&search_pipeline=cedarling_search'

The search_pipeline is required so the response to the query is intercepted and processed by the plugin which in turn will invoke Cedarling. The response will probably contain less hits than the query issued earlier and will come with an ext section that reports:

  • The amount of hits that passed authorization
  • The average decision time per hit

About development#

Once the work to get all of the pieces running is done, making changes to the plugin is rather straightforward: the Java code is in src directory and compilation is a matter of issuing ./gradlew clean compileJava.

In package-based installations, OpenSearch log is found at /var/log/opensearch/opensearch.log. To be able to see the logging statements produced by this plugin, add a line like the below to /etc/opensearch/opensearch.yml and restart opensearch (systemctl restart opensearch.service): 

logger.io.jans.cedarling: trace

Troubleshooting#

If gradle ever starts giving problems, try the following to "reset it":

rm -rf ~/.gradle/caches/*
rm -rf ~/.gradle/daemon/*