Using RediSearch
This blog covers how to use RediSearch in your .NETCore applications. It provides sample C# examples using the RediSearch NetCore client NRediSearch.
The article has two main sections:
- Creating RedisSearch records
- Searching for records.
Prerequisites
This examples in this blog use the following packages from nuget.org:
a) StackExchange.Redis -Version 2.1.58
b) NRediSearch - Version 3.1.0
c) Newtonsoft.Json -Version 12.0.3
The RediSearch samples were developed and tested with the versions shown; I imagine they'll work with later versions.
Sample Data
To make the examples a little more concrete, we created RediSearch records based on the SQL Server Northwind Database, scripts for which are here on GitHub.
To make your life easier, I have created a json file from the view Order Details Extended. If you want to follow along by creating your own Redis records your can download the json file northwindorders.json. There are 2155 records in the file. The records look like the json below.
[{"customerID":"VINET","customerName":"Vins et alcools Chevalier",
"address":"59 rue de l'Abbaye","city":"Reims", "postalCode":"51100","country":"France",
"orderID":10248,"orderDate":"1996-07-04T00:00:00",
"requiredDate":"1996-08-01T00:00:00","shippedDate":"1996-07-16T00:00:00",
"shipperName":"Federal Shipping","productID":11,"productName":"Queso Cabrales",
"unitPrice":14.0000,"quantity":12,"extendedPrice":168.0000,"id":null,"score":0},
{"customerID":"VINET","customerName":"Vins et alcools Chevalier",
"address":"59 rue de l'Abbaye","city":"Reims","postalCode":"51100","country":"France",
"orderID":10248,"orderDate":"1996-07-04T00:00:00",
"requiredDate":"1996-08-01T00:00:00","shippedDate":"1996-07-16T00:00:00",
"shipperName":"Federal Shipping","productID":42,"productName":"Singaporean Hokkien Fried Mee",
"unitPrice":9.8000,"quantity":10,"extendedPrice":98.0000,"id":null,"score":0}]Creating the Data
To create the records for RediSearch we'll use the POCO class shown below.
public class DocCommon
{
public string Id { get; set; }
public double Score { get; set; }
}
public class CustomerOrders : DocCommon
{
public string customerID { get; set; }
public string customerName { get; set; }
public string address { get; set; }
public string city { get; set; }
public string postalCode { get; set; }
public string country { get; set; }
public int orderID { get; set; }
public System.Nullable<System.DateTime> orderDate { get; set; }
public System.Nullable<System.DateTime> requiredDate { get; set; }
public System.Nullable<System.DateTime> shippedDate { get; set; }
public string shipperName { get; set; }
public int productID { get; set; }
public string productName { get; set; }
public decimal unitPrice { get; set; }
public short quantity { get; set; }
public System.Nullable<decimal> extendedPrice { get; set; }
}Note that the CustomerOrders class inherits DocCommon. The rational for this will be explained in a later section.
The first thing is to create a controller for handling this example.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System.IO;
using Microsoft.AspNetCore.Routing;
using System.Dynamic;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Cors;
using System.Reflection;
using System.Data.Common;
using System.Diagnostics;
//******** Needed for Redis Search *************
using Newtonsoft.Json;
using StackExchange.Redis;
using NRediSearch;
using static NRediSearch.Schema;
using NRediSearch.Aggregation;
using NRediSearch.QueryBuilder;
namespace RedisSearchApp.Controllers
{
[Produces("application/json")]
public class SearchController : ControllerBase
{
private readonly Microsoft.AspNetCore.Hosting.IWebHostEnvironment _hostingEnvironment;
private readonly IConnectionMultiplexer _connectionMultiplexer;
Client _rediSearchClient;
public SearchController(IConfiguration config, IWebHostEnvironment hostingEnvironment)
{
_hostingEnvironment = hostingEnvironment;
//In real life avoid hard-coding and use secure ways to specify
//your redis-server IP
string sconn = "127.0.0.1:6379";
_connectionMultiplexer = ConnectionMultiplexer.Connect(sconn);
IDatabase db = _connectionMultiplexer.GetDatabase();
_rediSearchClient = new Client("index_orders", db);
Schema schema = CreateSchema();
_rediSearchClient.CreateIndex(schema, new Client.ConfiguredIndexOptions(Client.IndexOptions.Default));
}
}The constructor in this example is responsible for:
- Establishing a connection to the Redis Server, using the .Net Redis Search Client (NRediSearch, courtesy - Marc Gravell and Nick Craver, StackExchange)
- Creating an index based on a provided schema.
Note: the index can be created only once; a RedisServerException will occur if an attempt is made to create an existing index. To avoid this, the code below is used to check for a pre-existing index.
Check if Index Exists
private async Task<bool> IndexExistsAsync(string checkIndexName)
{
InfoResult resultParsed = await _rediSearchClient.GetInfoParsedAsync();
string indexName = resultParsed.IndexName;
return (indexName == checkIndexName);
}We'll then change the last 2 lines of the constructor to:
if (!IndexExistsAsync("index_orders").Result)
{
Schema schema = CreateSchema();
_rediSearchClient.CreateIndex(schema, new Client.ConfiguredIndexOptions(Client.IndexOptions.Default));
}The code for CreateSchema is:
private Schema CreateSchema()
{
Schema schema = new Schema();
schema.AddTextField("customerID", 1)
.AddTextField("customerName",2)
.AddTextField("orderID", 1)
.AddTextField("city", 1)
.AddTextField("postalCode", 1)
.AddNumericField("extendedPrice")
.AddTextField("productName", 1)
};
return schema;
}Performing a Basic Search
Now that we have the preliminaries out of the way, let's do a basic search. for the word "Vins" – from the Northwind Database that we transferred to Redis. Shown below is the code used for the search.
[HttpGet]
[Route("api/search/redis/v.10/basic")]
public async Task<List<CustomerOrders>> SearchRedisOrdersBasic()
{
string searchString = "Vins";
SearchResult res = await _rediSearchClient.SearchAsync(new NRediSearch.Query(searchString) { WithPayloads = true });
List<CustomerOrders> customerOrdersList = new List<CustomerOrders>();
foreach (Document doc in res.Documents)
{
CustomerOrders _order = new CustomerOrders()
{
id = doc.Id,
score = doc.Score,
customerName = (string)doc["customerName"],
address = (string)doc["address"],
customerID = (string)doc["customerID"],
productName = (string)doc["productName"],
extendedPrice = Convert.ToDecimal(doc["extendedPrice"]),
//.. etc
};
customerOrdersList.Add(_order);
}
return customerOrdersList;
}Notes on the above code:
- If you have been following along with the sample data, you'll notice that 10 records are ruturned. We'll cover later how to control the number of records returned by the search query.
- We have hard-coded the search word "Vins"; in real-life you'll pass it as a parameter.
- Only the fields that are specified in the schema are searched for a match.
- In line 6, note that the query parameter WithPayloads is set to true. Failure to do that results in only the document id's being returned by the redis-server.
- All properties in the above example are cast to the appropriate types. In general you don't need to do this for string fields.
Improving the Version 1.0 api of this application to handle new properties.
Unlike a traditional relational database, Redis has no fixed schema for records. Consider the POCO class CustomerOrders in our example. The sample shown above will maintain the fixed properties we have have specified in the POCO. Now assuming after the app is released and populated with a vast number of records, we decide to add other fields to CustomerOders – such as ShipToAddress. To implement this, we'll have to modify codes after line #20 ( [see Route api/search/redis/v.10/basic and CustomerOrders to accommodate the changes. At best, this is a hassle; so let's explore how to improve on the previous approach.
The sample implementation below shows a simple way to side-step all the issues involved. It will require modifying only CustomerOrders. Once that is done, you'll have to save new records to include the ShipToAddress. We can then avoid headaches by using a json client (Newtonsoft.Json) to do the heavy lifting. The code below illustrates this.
[Route("api/search/redis/v2.0/basic")]
public async Task<List<CustomerOrders>> SearchRedisOrdersBasicUseJson()
{
string searchString = "Vins";
SearchResult res = await _rediSearchClient.SearchAsync(new NRediSearch.Query(searchString) { WithPayloads = true });
List<CustomerOrders> customerOrdersList = new List<CustomerOrders>();
foreach (Document doc in res.Documents)
{
IEnumerable<KeyValuePair<string, RedisValue>> record = doc.GetProperties();
string jsonRecord = JsonConvert.SerializeObject(record);
CustomerOrders custOrders = JsonConvert.DeserializeObject<CustomerOrders>(jsonRecord);
custOrders.id = doc.Id;
custOrders.score = doc.Score;
customerOrdersList.Add(custOrders);
}
return customerOrdersList;
}Only lines 12-22 (foreach bloc) are changed. The GetProperties method (line #11) returns a Redis KeyValuePair that we serialize and convert to CustomerOrders. This bypasses the hard coding and the casting issues of v1.0.
Wild Card Searches
The searches shown so far have been based on exact word searches. RediSearch also supports wildcard searches.
In the sample data used in this blog, one of the products is a product named Flotemysost, To get records for this product based on a wild card search, we can modify the example full search code to this:
//use an "*" to search for words that begin with "Flotem"
string searchString = "Flotem*";
SearchResult res = await _rediSearchClient.SearchAsync(new NRediSearch.Query(searchString) { WithPayloads = true });The example above will return records that have words that begin with "Flotem"