Stellar Cortex Click

JSONIndexer

JSONIndexer bulk indexes JSON documents into a catalog. It will attempt to automatically extract default values for documentId, url, and imageUrl, but provides utilities for customizing this mapping.

Example

const jsonArray = JSON.parse(fs.readFileSync(`./your-json.json`).toString());
 
const jsonIndexer = catalog.jsonIndexer(schemaItems, {
  getId: (item) => item.name,
});
 
await jsonIndexer.index();

Constructor

  • catalog: The catalog to index documents into.
  • documents: An array of JSON documents to be indexed.
  • opts: Optional configuration options.

JSONIndexerOpts

  • batchSize?: number: The number of documents to process in each batch (default: 50).
  • getId?: (document: any) => string: A function to extract the document ID (default: JSONIndexer.defaultGetId).
  • getUrl?: (document: any) => string | undefined: A function to extract the document URL (default: JSONIndexer.defaultGetUrl).
  • getImageUrl?: (document: any) => string | undefined: A function to extract the document image URL (default: JSONIndexer.defaultGetImageUrl).

Static methods

defaultGetId

static defaultGetId(document: any): string

Default method to extract the document ID. It looks for properties named "id", "Id", "ID", "documentId", "documentID", "DocumentId", or "DocumentID".

defaultGetUrl

static defaultGetUrl(document: any): string | undefined

Default method to extract the document URL. It looks for a property named "url".

defaultGetImageUrl

static defaultGetImageUrl(document: any): string | undefined

Default method to extract the document image URL. It looks for a property named "imageUrl".

Instance methods

index

async index(): Promise<void>

Indexes all documents in the provided array. This method processes the documents in batches and upserts them into the catalog.