Summary

The following examples assume you have downloaded the API client into your working directory.

Basic usage

If you already have created an index you'll need to use your index name to instantiate the client:

import com.flaptor.indextank.apiclient.IndexTankClient;
import com.flaptor.indextank.apiclient.IndexTankClient.Index;
import com.flaptor.indextank.apiclient.IndexTankClient.Query;

...

IndexTankClient client = new IndexTankClient("<YOUR API URL HERE>");
Index index = client.getIndex("<YOUR INDEX NAME HERE>");

Once you have an instance of the client all you need is the content you want to index.
The simplest way to add a document is sending that content in a single field called "text":

String documentId = "<YOUR DOCUMENT ID>";
String documentText = "<THE TEXTUAL CONTENT>";

Map<String, String> fields = new HashMap<String, String>();
fields.put("text", documentText);

index.addDocument(documentId, fields);

That's it, you have indexed a document.

You can now search the index for any indexed document by simply providing the search query:

import com.flaptor.indextank.apiclient.IndexTankClient.SearchResults;

...

String query = "<YOUR QUERY STRING>";

SearchResults results = index.search(Query.forString(query));

System.out.println("Matches: " + results.matches);

for (Map<String, Object> document : results.results) {
    System.out.println("doc id: " + document.get("docid"));
}

As you can see, the results are the document ids you provided when indexing the documents. You use them to retrieve the actual documents from your DB to render the result page.

You can get richer results using the fetch fields and snippet fields options:

results = index.search(Query.forString(query) 
                        .withSnippetFields("text") 
                        .withFetchFields("title", "timestamp"));


System.out.println("Matches: " + results.matches);
for (Map<String, Object> document : results.results) {
    System.out.println("id: " + document.get("docid") + 
                       ";title: " + document.get("title") +
                       ";timestamp: " + document.get("timestamp") +
                       ";snippet: " + document.get("snippet_text"));
}

Deleting an indexed document is also very easy:

String documentId = "<YOUR DOCUMENT ID>";

index.deleteDocument(documentId);

Additional fields [back to top]

When you index a document you can define different fields by adding more elements to the document object:

// INDEX MULTIPLE FIELDS
String title = "<DOCUMENT TITLE>";
String author = "<DOCUMENT AUTHOR>";

fields = new java.util.HashMap<String, String>();
fields.put("text", documentText);
fields.put("title", title);
fields.put("author", author);

index.addDocument(documentId, fields);

By default, searches will only look at the "text" field, but you can define special searches that look at other fields by prefixing a search term with the field name. The following example filters results including only the user's content.

// FILTER TO USER'S CONTENT
results = index.search(Query.forString(query + " author:" + author));

There's also a special field named "timestamp" that is expected to contain the publication date of the content in seconds since unix epoch (1/1/1970 00:00 UTC). If none is provided, the time of indexation will be used by default.

fields.put("timestamp", 
            Long.toString(System.currentTimeMillis() / 1000L));
index.addDocument(documentId, fields);

Document variables [back to top]

When you index a document you can define special floating point fields that can be used in the results scoring. The following example can be used in an index that allows 3 or more variables:

// INDEX DOCUMENT WITH VARIABLES
Float rating = 0.5f;
Float reputation = 1.5f;
Float visits = 10.0f;

Map<Integer, Float> variables = new HashMap<Integer, Float>();
variables.put(0, rating);
variables.put(1, reputation);
variables.put(2, visits);

index.addDocument(documentId, fields, variables);

You can also update a document variables without having to re-send the entire document. This is much faster and should always be used if no changes were made to the document itself.

// UPDATE DOCUMENT VARIABLES ONLY
variables.put(0, newRating);
variables.put(2, newVisits);

index.updateVariables(documentId, variables);

Scoring functions [back to top]

To use the variables in your searches you'll need to define scoring functions. These functions can be defined in your dashboard by clicking "Manage" in your index details or directly through the API client.

// FUNCTION 0 : sorts by most recent
index.addFunction(0, "-age");

// FUNCTION 1 : standard textual relevance
index.addFunction(1, "relevance");

// FUNCTION 2 : sorts by rating
index.addFunction(2, "doc.var[0]");

// FUNCTION 3 : sorts by reputation
index.addFunction(3, "d[1]"); 

// FUNCTION 4 : advanced function
index.addFunction(4, "log(d[0]) - age/50000");

Read the function definition syntax for help on how to write functions.

If you don't define any functions, you will get the default function 0 which sorts by timestamp (most recent first). By default, searches will use the function 0. You can specify a different function when searching by using the option scoring_function

index.search(Query.forString(query).withScoringFunction(2));

Query variables and Geolocation[back to top]

Besides the document variables, in the scoring functions you can refer to query variables. These variables are defined at query time and can be different for each query.

A common use-case for query variables is geolocation, where you use two variables for latitude and longitude both in the documents and in the query, and use a distance function to sort by proximity to the user. For this example will assume that every document stores its position in variables 0 and 1 representing latitude and longitude respectively.

Defining a proximity scoring function:

// FUNCTION 5 : inverse distance calculated in miles
index.addFunction(5, "-mi(d[0], d[1], q[0], q[1])");

Searching from a user's position:

results = index.search(Query.forString(query)
                   .withScoringFunction(5)
                   .withQueryVariable(0, latitude)
                   .withQueryVariable(1, longitude)); 

Faceting[back to top]

Besides being able to define numeric variables on a document you can tag documents with category values. Each category is defined by string, and its values are alse defined by strings. So for instance, you can define a category named "articleType" and its values can be "camera", "laptop", etc... You can have another category called "priceRange" and its values can be "$0 to $49", "$50 to $100", etc...

Documents can be tagged with a single value for each category, so if a document is in the "$0 to $49" priceRange it can't be in any other, and retagging over the same category results in overwriting the value.

You can tag several categories at once like this:

Map<String, String> categories = new HashMap<String, String>();
categories.put("priceRange", "$0 to $299");
categories.put("articleType", "camera");

index.updateCategories(docId, categories); 

When searching, you will get an attribute in the results called "facets", and it will contain a dictionary with categories for keys. For each category the value will be another map, with category value as key and occurences as value. So for instance:

Map<String, Map<String, Integer>> facets = results.facets;

...

{ 
    'articleType': {
        'camera': 5,
        'laptop': 3
    },
    'priceRange': {
        '$0 to $299': 4,
        '$300 to $599': 4
    }
}

Means that from the matches, 5 are of the "camera" articleType and 3 are "laptop". Also, 4 of them all are in the "$0 to $299" priceRange, and 4 on the "$300 to $599".

Then, you can also filter a query by restricting it to a particular set of category/values. For instance the following will only return results that are of the "camera" articleType and also are either in th "$0 to $299" or "$300 to $599" price range.

Map<String, List<String>> filters = 
        new HashMap<String, List<String>>();
filters.put("priceRange", 
            Arrays.asList("$0 to $299", "$300 to $599"));
filters.put("articleType", 
            Arrays.asList("camera"));

results = index.search(Query.forString(query)
                        .withCategoryFilters(filters));

Range queries [back to top]

Document variables and scoring functions can also be used to filter your query results. When performing a search it is possible to add variable and function filters. This will allow you to only retrieve, in the search results, documents whose variable values are within a specific range (e.g.: posts that have more than 10 votes but less than a 100). Or only return documents for the which a certain scoring function returns values within a specific range.

You can specify more than one range for each variable or function (the value must be within at least ONE range) filter, and you can use as many filters as you want in every search (all filters must be met):

/*
  In this sample, the results will only include documents 
  whose variable #0 value is between 5 and 10 or between 15
  and 20, and variable #1 value is less than or equal to 3
*/
results = index.search(Query.forString(query)
           .withDocumentVariableFilter(0, 5d, 10d)
           .withDocumentVariableFilter(0, 15d, 25d)
           .withDocumentVariableFilter(1, Double.NEGATIVE_INFINITY, 3));

// The same applies to functions
results = index.search(Query.forString(query)
           .withFunctionFilter(0, 0.5d, Double.POSITIVE_INFINITY)

Batch indexing [back to top]

When populating an index for the first time or when a batch task for adding documents makes sense, you can use the batch indexing call.

When using batch indexing, you can add a large batch of documents to the Index with just one call. There is a limit to how many documents you can add in a single call, though. This limit is not related to the number of documents, but to the total size of the resulting HTTP request, which should be less than 1MB.

Making a batch indexing call reduces the number of request needed (reducing the latency introduced by round-trips) and increases the maximum throughput which can be very useful when initially loading a large index.

The indexing of individual documents may fail and your code should handle that and retry indexing them. If there are formal errors in the request, the entire batch will be rejected with an exception.

In the JAVA client, you can use a convenient Document object for the documents and you can pass a simple Iterable to the addDocuments() method.

List<Document> documents = new ArrayList<Document>();

Map<String, String> fields1 = new HashMap<String, String>();
fields1.put("text", "document 1 text");

// Document is built with:
// - String docId
// - Map<String, String> fields
// - Map<Integer, Float> variables 
// - Map<String, String> facetingCategories
Document document1 = new Document("1", fields1, null, null);
documents.add(document1); 

Map<String, String> fields2 = new HashMap<String, String>();
fields1.put("text", "document 2 text");
Map<Integer, Float> variables2 = new HashMap<Integer, Float>();
variables2.put(1, 0.4f);
Document document2 = new Document("2", fields2, variables2, null);
documents.add(document2); 

BatchResults results = index.addDocuments(documents);

The results object provides you with a way to retry when some of the documents failed to be indexed:

Iterable<Document> failedDocuments = results.getFailedDocuments();
index.addDocuments(failedDocuments);

Bulk Delete [back to top]

With this method, you can delete a batch of documents (reducing the latency introduced by round-trips). The total size of the resulting HTTP request should be less than 1MB.

The deletion of individual documents may fail and your code should handle that and retry deleting them. If there are formal errors in the request, the entire batch will be rejected with an exception.

List<String> docids = new ArrayList<String>();
docids.add("doc1");
docids.add("doc2");
docids.add("doc3");
docids.add("doc4");

BulkDeleteResults results = index.deleteDocuments(docids);

The response will be an array with the same length as the sent batch. Each element will be a map with the key "deleted" denoting whether the document with the id in this position of the batch was successfully deleted from the index. If it's false, an error message will also be in the map with the key "error".

Iterable<String> failedDocids = results.getFailedDocids();
index.deleteDocuments(failedDocuments);

Delete by Search [back to top]

With this method, you can delete a batch of documents that match a particular search query. You can use many of the same arguments applied to a normal search - start (which will preserve the results found before the value of start), scoring function, category filters, variables, and docvar filters.

index.deleteBySearch(Query.forString(query))

Index management [back to top]

You can create and delete indexes directly with the API client. These methods are equivalent to their corresponding actions in the dashboard. Keep in mind that index creation may take a few seconds.

The create_index methods will return the new index's client:

// this parameter allows you to create indexes with public search enabled.
// default is false. 

boolean publicSearchEnabled = false;
Index index = client.createIndex("test_name", publicSearchEnabled);

while (!index.hasStarted()) {
  Thread.sleep(300);
}

The delete_index method completely removes the index represented by the object.

client.deleteIndex("test_name");