# Neural Search Quickstart for OpenSearch ## **Intro** Neural Search is now generally available as of OpenSearch version 2.9.0. The Neural Search plugin allows the use of pre-trained models for indexing text segments to existing k-NN enabled indices, as well as transforming queries into the same vector space. You don’t need to train any models yourself, and you don’t need any specialized hardware. All you need is Docker (well, at least in order to get up and running in five minutes following this guide). Dependencies for this project * Docker * [**https://github.com/Npfries/opensearch-neural-search**](https://github.com/Npfries/opensearch-neural-search) ## **Neural Search** Neural Search is OpenSearch’s end to end vector embedding and search solution, leveraging existing k-NN search features and off-the-shelf NLP models for vector encoding. This example is using the `huggingface/sentence-transformers/all-MiniLM-L12-v2` model, which transforms sentences and paragraphs into 384 dimensional vector space. Suppose we have a list of movie titles, and would like to provide some basic semantic search capability. Historically, this would be achieved by having high quality metadata available for searching based on relevance. Descriptions, lists of actors, genres, etc. would be needed to ensure high quality search results. In this example, we are working with a *lot* less data, yet the semantic search capability is still quite usable for many applications. For example, the following list of movies are inserted into the `movies`index. Note that the only information we have available is the move title. ```plaintext POST /movies/_bulk { "index": {} } { "title": "Star Wars" } { "index": {} } { "title": "Lord of the Rings" } { "index": {} } { "title": "Spiderman" } { "index": {} } { "title": "Indiana Jones" } ``` Here is an example search performed using the “neural\_search\_template” search template. ```plaintext GET /movies/_search/template { "id": "neural_search_template", "params": { "query": "lightsaber", "from": 0, "size": 1 } } ``` In the result, we can see that “Star Wars” is the top hit. If you are familiar with Elasticsearch or OpenSearch, but haven’t used Neural Search or another means of NLP and k-NN search, this might be surprising. ```plaintext { "took": 40, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 7, "relation": "eq" }, "max_score": 0.5106329, "hits": [ { "_index": "movies", "_id": "ZluZuIkBNHby32LgCK2-", "_score": 0.5106329, "fields": { "title": [ "Star Wars" ] } } ] } } ``` ## **Starting OpenSearch** Here is an example Docker Compose file that configures one master node as a data node, and one dedicated ML node. This configuration should be used for development purposes only, it is not a production ready configuration. This file is provided in the [**example repo**](https://github.com/Npfries/opensearch-neural-search) if you prefer to pull it instead of creating your own docker-compose.yml file. ```plaintext version: '3.9' services: opensearch-node1: image: opensearchproject/opensearch:2.9.0 container_name: opensearch-data-node environment: - cluster.name=opensearch-cluster - node.name=opensearch-node1 - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1 - bootstrap.memory_lock=true - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" - "DISABLE_INSTALL_DEMO_CONFIG=true" - "DISABLE_SECURITY_PLUGIN=true" ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 volumes: - opensearch-data1:/usr/share/opensearch/data ports: - 9200:9200 - 9600:9600 networks: - opensearch-net opensearch-node2: image: opensearchproject/opensearch:2.9.0 container_name: opensearch-ml-node environment: - cluster.name=opensearch-cluster - node.name=opensearch-node2 - node.roles=ml - discovery.seed_hosts=opensearch-node1,opensearch-node2 - cluster.initial_cluster_manager_nodes=opensearch-node1 - bootstrap.memory_lock=true - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" - "DISABLE_INSTALL_DEMO_CONFIG=true" - "DISABLE_SECURITY_PLUGIN=true" ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536 volumes: - opensearch-data2:/usr/share/opensearch/data networks: - opensearch-net opensearch-dashboards: image: opensearchproject/opensearch-dashboards:latest container_name: opensearch-dashboards ports: - 5601:5601 expose: - "5601" environment: - 'OPENSEARCH_HOSTS=["http://opensearch-node1:9200","http://opensearch-node2:9200"]' - "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" networks: - opensearch-net volumes: opensearch-data1: opensearch-data2: networks: opensearch-net: ``` Assuming Docker is installed on your system, it should start with the following command. ```plaintext docker compose up -d ``` This should start three containers: * opensearch-dashboards * opensearch-data-node * opensearch-ml-node ## **Uploading an ML Model** For this example, we will be using an off-the-shelf model provided on Hugging Face: `huggingface/sentence-transformers/all-MiniLM-L12-v2`. We need to upload the model to OpenSearch. Open OpenSearch Dashboards at [`http://localhost:5601`](http://localhost:5601) and navigate to Dev Tools. All queries are also listed in `queries.txt` in the [**example repo**](https://github.com/Npfries/opensearch-neural-search) for convenience. We can use the following request to upload the model to our OpenSearch host. ```plaintext POST /_plugins/_ml/models/_upload { "name": "huggingface/sentence-transformers/all-MiniLM-L12-v2", "version": "1.0.1", "model_format": "TORCH_SCRIPT" } ``` The response should look something like this: ```plaintext { "task_id": "-97CuIkBWLo6ADqAT1Tn", "status": "CREATED" } ``` Note your task ID! We’re going to use it to check on the status, as it is an asynchronous operation. We need to wait for this to complete before proceeding. Your task ID will be different, you should replace ``with yours. Run the following command to check the status of the upload. ```plaintext GET /_plugins/_ml/tasks/ ``` When the `state` of the task status says `COMPLETED` you should be able to copy the `model_id`. The `model_id` is going to be used several times. ```plaintext { "model_id": "JHrCuIkBkyiJw75fUf-n", "task_type": "REGISTER_MODEL", "function_name": "TEXT_EMBEDDING", "state": "COMPLETED", "worker_node": [ "VF5J2Z00QNGLfL4l6c5e1w" ], "create_time": 1691021889469, "last_update_time": 1691021945252, "is_async": true } ``` The model now needs to be loaded. Run the following command, replacing `` with the `model_id` from the resulting task. ```plaintext POST /_plugins/_ml/models//_load ``` This is another asynchronous operation, with another `task_id`. Repeat the task command with the new `task_id`. ```plaintext GET /_plugins/_ml/tasks/ ``` When that task is completed, we can proceed with creating an ingest pipeline that will use this model. The `model_id` will not change after uploading. ## **Creating the Ingest Pipeline** The following command will create an ingest pipeline names `title-embedding`that uses the model uploaded above. ```plaintext PUT _ingest/pipeline/title-embedding { "description": "Title embedding pipeline", "processors" : [ { "text_embedding": { "model_id": "", "field_map": { "title": "title_embedding" } } } ] } ``` The `field_map` property describes the relationship between the text field that is provided, in this case is it movie titles, so I am using the `title`field. The `title_embedding` field is where the ingest pipeline will place the output of the model. The `field_map` can contain as many mappings as needed. Multiple models can be used by including more than one entry for `processors`. This ingest pipeline can be used for more than one index, given that the `field_map` is applicable to both. It is currently not used, we need to tell OpenSearch to use this pipeline when indexing to one or more indices. This must be defined on the index settings. ## **Creating an Index** Use the following command to create an index that uses the `title-embedding`ingest pipeline. ```plaintext PUT /movies { "settings": { "index.knn": true, "default_pipeline": "title-embedding" }, "mappings": { "properties": { "title_embedding": { "type": "knn_vector", "dimension": 384, "method": { "name": "hnsw", "engine": "lucene" } }, "title": { "type": "text" } } } } ``` We have to supply some information in the index settings and field mapping for OpenSearch to understand how to process and index documents. * `index.knn: true` enables k-NN search for this index. * `default_pipeline: title-embedding` specifies the ingest pipeline which should be used. * `type: knn_vector` is the field mapping type for `title_embedding` * `dimension: 384` refers to the *dimensionality* of the model chosen. In this case, it is 384. * `method` in the future, other methods may be used, but these are currently supported by OpenSearch Technically, we could stop here, index some movies, and search by specifying the model using a `neural` query, but that would require providing the `model_id` in each search request. This is less than ideal, as in most cases, we will want to ensure we are using the same model that was used for indexing. We will create a search template to make this easier. ## **Creating a Search Template** The following command will create a search template named `neural_search_template`. Make sure to replace the `` with the `model_id` from earlier. ```plaintext POST _scripts/neural_search_template { "script": { "lang": "mustache", "source": { "from": "{{from}}{{^from}}0{{/from}}", "size": "{{size}}{{^size}}10{{/size}}", "query": { "neural": { "title_embedding": { "query_text": "{{query}}", "model_id": "", "k": 200 } } }, "fields": [ "title" ], "_source": false } } } ``` Note the `neural` query. This is a new type of query which allows inline use of an ML model for encoding the `query_text` to be used when searching using k-NN on the `title_embedding` field. \*Note: If you’d like to learn about managing index mappings and search templates in production, you can read more about how I [**do it using migrations**](https://nathanfries.com/posts/opensearch-migrations/). ## **Adding Documents** As promised above, populate the movies index with some data. Note that documents contain only the title field. ```plaintext POST /movies/_bulk { "index": {} } { "title": "Star Wars" } { "index": {} } { "title": "Lord of the Rings" } { "index": {} } { "title": "Spiderman" } { "index": {} } { "title": "Indiana Jones" } ``` If everything worked as expected, querying this index using a `neural` query or utilizing the search template above should return results based on the semantic similarity of the search term and the title. ## **Querying** Here is an example query using the search template created above. ```plaintext GET movies/_search/template { "id": "neural_search_template", "params": { "query": "lightsaber", "from": 0, "size": 1 } } ``` The expected result is that a single result is returned, and it should be the one that most closely relates to the search term. ```plaintext { "took": 46, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 4, "relation": "eq" }, "max_score": 0.5106329, "hits": [ { "_index": "movies", "_id": "_d5tuYkBWLo6ADqA4lQb", "_score": 0.5106329, "fields": { "title": [ "Star Wars" ] } } ] } } ``` The result makes sense. “Lightsaber” is most closely related to “Star Wars” in the list of movies. We can continue querying to confirm it is working. Searching “web” returns “Spiderman”, “dragon” returns “Lord of the Rings” and so on. Hopefully, this is enough information to understand the basics of the Neural Search feature, and whether or not it would be useful for your applications. This is not intended to be an in-depth guide to k-NN search. If you are looking for more information on that topic, I highly recommend [**An Introduction to Statistical Learning**](https://www.statlearning.com/) published by Springer.