Full Text Search with ASP.NET MVC, jQuery autocomplete and Elasticsearch

gogo8848 2015-07-01

展开全文

This article demonstrates how to do a full text search using jQuery Autocomplete with an ASP.NET MVC application and an Elasticsearch search engine. CRUD operations are also implemented for Elasticsearch ( ElasticsearchCRUD ).

To download and install Elasticsearch, use the instructions here .

Code: https://github.com/damienbod/WebSearchWithElasticsearch

Search for documents in Elasticsearch

The application uses ElasticsearchCRUD to access Elasticsearch. The API can use any DTO or entity class and this is automatically mapped to an Elasticsearch index and type using the class Type. The default index is the type of the class pluralized and all characters are converted to lower case characters. The default type is the type name in lower case characters. This can easily be changed if a different mapping is required for Elasticsearch. For example, classes which are saved as child documents in Elasticsearch need a mapping for its index, ie. use the index where the parent document is stored.

A Skill class is used in this example. This class is saved in the search engine with the index ‘skills’ and the type ‘skill’.

public class Skill
{
  [Required]
  [Range(1, long.MaxValue)]
  public long Id { get; set; }
    
  [Required]
  public string Name { get; set; }
    
  [Required]
  public string Description { get; set; }
    
  public DateTimeOffset Created { get; set; }
    
  public DateTimeOffset Updated { get; set; }
}

The ElasticsearchCRUD Search method is used to do the full text search using QueryString. The method takes the term(s) from the SearchController and does a QueryString search in the engine. This search is case insensitive.

You can do almost any search with context.Search method, not just a query string search. Highlighted results are not supported and multiple index or multiple type searches are not supported. All you need to do is build a JSON query for the search and a collection of types is returned from the hits. The Query is built using the StringBuilder class, this is not very eloquent, but it’s sufficient for this level of complexity. If your search logic is more complex, you would require a better way of creating the queries. One possibility would be to use NEST.

public IEnumerable<Skill> QueryString(string term)
{
  return _context.Search<Skill>(BuildQueryStringSearch(term));
}

private string BuildQueryStringSearch(string term)
{
  var names = "";
  if (term != null)
  {
    names = term.Replace("+", " OR *");
  }

  var buildJson = new StringBuilder();
  buildJson.AppendLine("{");
  buildJson.AppendLine(" \"query\": {");
  buildJson.AppendLine("   \"query_string\": {");
  buildJson.AppendLine("      \"query\": \"" + names  + "*\"");
  buildJson.AppendLine("     }");
  buildJson.AppendLine("  }");
  buildJson.AppendLine("}");

  return buildJson.ToString();
}

The search method is used in the MVC SearchController. Json is returned because the action method is used by the jQuery Autocomplete control.

public JsonResult Search(string term)
{			
  return Json(_searchProvider.QueryString(term), "Skills", JsonRequestBehavior.AllowGet);
}

The Autocomplete (jquery-ui) source method uses the action method search from the SearchController. This saves the Object array to the autocomplete control. This control requires both a label and a value property in each item for the control. The select method is used to choose a result from the search. The selected item is then added to other html controls on the page for update or delete HTTP requests.

<fieldset class="form">
  <legend>SEARCH for a document in the search engine</legend>
  <table width="500">
    <tr>
      <th></th>
    </tr>
    <tr>
      <td>
        <label for="autocomplete">Search: </label>
      </td>
    </tr>
    <tr>
      <td>
        <input id="autocomplete" type="text" style="width:500px" />
      </td>
    </tr>
  </table>
</fieldset>

@section scripts
{
  <script type="text/javascript">
    var items;
    $(document).ready(function() {
      $("input#autocomplete").autocomplete({
        source: function(request, response) {
          $.ajax({
            url: "search",
            dataType: "json",
            data: {
              term: request.term,
            },
            success: function(data) {
              var itemArray = new Array();
              for (i = 0; i < data.length; i++) {
                itemArray[i] = { label: data[i].Name, value: data[i].Name, data: data[i] }
              }
 
              console.log(itemArray);
              response(itemArray);
            },
            error: function(data, type) {
              console.log(type);
            }
          });
        },
        select: function (event, ui) {
          $("#spanupdateId").text(ui.item.data.Id);
          $("#spanupdateCreated").text(new Date(parseInt(ui.item.data.Created.substr(6))));
          $("#spanupdateUpdated").text(new Date(parseInt(ui.item.data.Updated.substr(6))));

          $("#updateName").text(ui.item.data.Name);
          $("#updateDescription").text(ui.item.data.Description);
          $("#updateName").val(ui.item.data.Name);
          $("#updateDescription").val(ui.item.data.Description);

          $("#updateId").val(ui.item.data.Id);
          $("#updateCreated").val(ui.item.data.Created);
          $("#updateUpdated").val(ui.item.data.Updated);

          $("#spandeleteId").text(ui.item.data.Id);
          $("#deleteId").val(ui.item.data.Id);
          $("#deleteName").text(ui.item.data.Name);

          console.log(ui.item);
        }
      });
    });
</script>
}

The control can then be used as follows:

Elasticsearch Create, Update and Delete

To make it easy to do searches and provide some data, Create, Update and Delete implementations have been added using ElasticsearchCRUD. The Provider uses the context from ElasticsearchCRUD and executes all pending changes in one single bulk request to Elasticsearch. Multiple indexes, types can be executed in one single context, bulk request. The HttpClient class is used under the hub of the context. This works well when adding or changing large amounts of data in Elasticsearch.

private const string ConnectionString = "http://localhost:9200/";
private readonly IElasticsearchMappingResolver _elasticsearchMappingResolver = new ElasticsearchMappingResolver();

public void AddUpdateEntity(Skill skill)
{
   using (var context = new ElasticsearchContext(ConnectionString, _elasticsearchMappingResolver))
   {
     context.AddUpdateDocument(skill, skill.Id);
     context.SaveChanges();
   }
}

public void UpdateSkill(long updateId, string updateName, string updateDescription)
{
  using (var context = new ElasticsearchContext(ConnectionString, _elasticsearchMappingResolver))
  {
    var skill = context.GetDocument<Skill>(updateId);
    skill.Updated = DateTime.UtcNow;
    skill.Name = updateName;
    skill.Description = updateDescription;
    context.AddUpdateDocument(skill, skill.Id);
    context.SaveChanges();
  }
}

public void DeleteSkill(long deleteId)
{
   using (var context = new ElasticsearchContext(ConnectionString, _elasticsearchMappingResolver))
   {
     context.DeleteDocument<Skill>(deleteId);
     context.SaveChanges();
   }
}

This can then be used in the SearchController. Validation has been implemented for the Create skill document in the HTTP Post Index action method. This calls the AddUpdateDocument method of ElasticsearchCRUD. If a document with the same Id already exists in the search engine, the document is overwritten. Usually the entities come from a primary database and are saved the elasticsearch for optimal searches. The primary database manages the Ids of the entity so this is not a problem for this use case.

readonly ISearchProvider _searchProvider = new ElasticSearchProvider();

[HttpGet]
public ActionResult Index()
{
  return View();
}

[HttpPost]
public ActionResult Index(Skill model)
{
  if (ModelState.IsValid)
  {
   model.Created = DateTime.UtcNow;
   model.Updated = DateTime.UtcNow;
   _searchProvider.AddUpdateDocument(model);

   return Redirect("Search/Index");
  }

  return View("Index", model);
}

[HttpPost]
public ActionResult Update(long updateId, string updateName, string updateDescription)
{
  _searchProvider.UpdateSkill(updateId, updateName, updateDescription);
  return Redirect("Index");
}

[HttpPost]
public ActionResult Delete(long deleteId)
{
  _searchProvider.DeleteSkill(deleteId);
  return Redirect("Index");
}

The controller can then be used in the view. The view has 3 different forms: Create, Delete and Update. The Delete and Update forms are updated, when a search result is selected from the Autocomplete control.

<form name="input" action="update" method="post">
  <fieldset class="form">
    <legend>UPDATE an existing document in the search engine</legend>
    <table width="500">
      <tr>
        <th></th>
        <th></th>
      </tr>
      <tr>
        <td>
          <span>Id:</span>
        </td>
        <td>
          <span id="spanupdateId">-</span>
          <input id="updateId" name="updateId" type="hidden" />
        </td>
      </tr>
      <tr>
        <td>
          <span>Name:</span>
        </td>
        <td>
          <input id="updateName" name="updateName" type="text" />
        </td>
      </tr>
      <tr>
        <td>
          <span>Description:</span>
        </td>
        <td>
          <input id="updateDescription" name="updateDescription" type="text" />
        </td>
      </tr>
      <tr>
        <td>
          <span>Created:</span>
        </td>
        <td>
          <span id="spanupdateCreated">-</span>
          <input id="updateCreated" name="updateCreated" type="hidden" />
        </td>
      </tr>
      <tr>
        <td>
          <span>Updated:</span>
        </td>
        <td>
          <span id="spanupdateUpdated">-</span>
          <input id="updateUpdated" name="updateUpdated" type="hidden" />
        </td>
      </tr>
      <tr>
        <td>
          <br />
          <input type="submit" value="Update Skill" style="width: 200px" />
        </td>
        <td></td>
      </tr>
    </table>
  </fieldset>
</form>

<form name="input" action="delete" method="post">
  <fieldset class="form">
    <legend>DELETE an existing document in the search engine</legend>
    <table width="500">
      <tr>
        <th></th>
        <th></th>
      </tr>
      <tr>
        <td>
          <span id="deleteName">-</span>
        </td>
        <td>
          <span id="spandeleteId">-</span>
          <input id="deleteId" name="deleteId" type="hidden" />
        </td>
      </tr>

      <tr>
        <td>
          <br />
          <input type="submit" value="Delete Skill" style="width: 200px" />
        </td>
        <td></td>
      </tr>
    </table>
  </fieldset>
</form>

@using (Html.BeginForm("Index", "Search"))
{
  @Html.ValidationSummary(true)

  <fieldset class="form">
    <legend>CREATE a new document in the search engine</legend>
    <table width="500">
      <tr>
        <th></th>
        <th></th>
      </tr>
      <tr>
        <td>
          @Html.Label("Id:")
        </td>
        <td>
          @Html.EditorFor(model => model.Id)
          @Html.ValidationMessageFor(model => model.Id)
        </td>
      </tr>
      <tr>
        <td>
          @Html.Label("Name:")
        </td>
        <td>
          @Html.EditorFor(model => model.Name)
          @Html.ValidationMessageFor(model => model.Name)
        </td>
      </tr>
      <tr>
        <td>
          @Html.Label("Description:")
        </td>
        <td>
          @Html.EditorFor(model => model.Description)
          @Html.ValidationMessageFor(model => model.Description)
        </td>
      </tr>
      <tr>
        <td>
          <br />
          <input type="submit" value="Add Skill" style="width:200px" />
        </td>
        <td></td>
      </tr>
    </table>
  </fieldset>
}

The application can then be used in the browser as follows:

Conclusion

As you can see, it is very easy to do a full text search using ASP.NET MVC with Elasticsearch. This could very easily be changed to use Web API with Angular JS. As these scale well, and have great performance, you could use this for almost any application.

Full Text Search with ASP.NET MVC, jQuery autocomplete and Elasticsearch

Other tutorials:

Search for documents in Elasticsearch

Elasticsearch Create, Update and Delete

Conclusion

Links: