NAME

Kernel::System::DocumentSearch::IndexManagement – Backend to manage Elasticsearch indices.

PUBLIC INTERFACE

Provides methods to manage the handling of indices and their related documents. The index management backend is designed to gather needed information about index settings, index mappings, document listings and document data about the different kinds of documents (tickets, articles etc.) from index driver modules, which knows about the data related to their particular data type. Those different types of data, that are handled by the index management are called DocumentTypes.

Such a DocumentType is used to determine the correct index driver and therefore gather the correct type of data. An index basically consists of an index name, a document type, related index settings, a mapping to define the available fields, their types and further behaviors, possible indexing pipelines and documents that might be handled.

For a detailed list of possible index settings, please refer to https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-modules-settings.

For a detailed list of possible mapping types and configurations, please refer to: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html

The documents of every index driver will be stored in their own, separated index and since the different documents might have the same ID's (compared to other document types), the index management handles documents every time with their DocumentType and DocumentID to have a unique access to every single data set.

The different index driver modules are located below Kernel::System::DocumentSearch::Driver::*, please refer to the single modules for more information about their usage.

IndexVerify()

Method that verifies, if the index settings and index mappings differs between the data provided by the index drivers and the data stored on server side, for a particular DocumentType:

    my $Result = $IndexManagementObject->IndexVerify(
        DocumentType => 'Ticket',
    );

Returns:

    {
        Success  => 1,
        Settings => {
            Valid   => 1,
            Invalid => [],
        },
        Mappings => {
            Valid   => 1,
            Invalid => [],
        },
    }

or in case of errors something like:

    {
        Success  => 0,
        Settings => {
            Valid   => 1,
            Invalid => [],
        },
        Mappings => {
            Valid   => 0,
            Invalid => [ 'QueueID' ],
        },
    }

IndexCreate()

Method that creates empty indices on server side, including their related index settings and mappings:

    my $Success = $IndexManagementObject->IndexCreate(
        DocumentType => 'Ticket',
    );

Returns 1 if the index creation was successful and 0, if not.

IndexRebuild()

Method that marks documents for rebuild:

    my $Success = $IndexManagementObject->IndexRebuild(
        DocumentType => 'Ticket',
        DocumentID   => [ 1, 2, 3 ] # optional, can be undef, a single id or an array reference of id's.
                                    # If undef, all documents by a certain document type will be marked.
    );

Returns 1 if the mark action was successful and 0, if not.

IndexDelete()

Method that deletes an entire index, including it's documents:

    my $Success = $IndexManagementObject->IndexDelete(
        DocumentType    => 'Ticket',
        RemovePipelines => 1,           # optional, defaults to 1.
    );

Returns 1 if the index deletion was successful and 0, if not.

IndexRefresh()

Method that manually refreshes an index to make new documents searchable:

    my $Success = $IndexManagementObject->IndexRefresh(
        DocumentType => 'Ticket',
    );

Returns 1 if the refresh action was successful and 0, if not.

Note: Automatic refreshing is done by Elasticsearch and it usually takes 1 second

PipelineCreate()

Method that create or update a set of pipelines.

    my $Success = $IndexManagementObject->PipelineCreate(
        Pipelines => {
            'SomePipeline' => {
                description => 'some description',
                processors  => [
                    {
                        set => {
                            field => 'some field',
                            value => "some value",
                        },
                    },
                    # ...
                ]
            },
            # ...
        }
    );

Returns 1 if the create action was successful and 0, if not.

PipelineRemove()

Method that removes a set of pipelines.

    my $Success = $IndexManagementObject->PipelineRemove(
        Pipelines => {
            'SomePipeline' => {
                description => 'some description',
                processors  => [
                    {
                        set => {
                            field => 'some field',
                            value => "some value",
                        },
                    },
                    # ...
                ]
            },
            # ...
        }
    );

Returns 1 if the remove action was successful and 0, if not.

PipelineVerify()

Method that checks a single given pipeline data gather from the server against the definition in the module file.

    my $Success = $IndexManagementObject->PipelineVerify(
        PipelineType => 'SomePipeline',                 # pipeline ID without the 'otrs_' prefix
        PipelineBody => {                               # pipeline definition from the server
            description => 'some description',
            processors  => [
                {
                    set => {
                        field => 'some field',
                        value => "some value",
                    },
                },
                # ...
            ]
        },
    );

Returns 1 if the data is equals and 0, if not.

PipelineSync()

Method that removes all OTRS pipelines from the server that does not have a definition module, updates any pipeline with different information than the provided by the definition modules and creates any missing one.

    my $Success = $IndexManagementObject->PipelineSync();

Returns 1 if the sync operation was successful and 0, if not.

DocumentStatus()

Method that returns a status about the indexed and available documents for a certain index:

    my $Status = $IndexManagementObject->DocumentStatus(
        DocumentType => 'Ticket',
    );

Returns:

    {
        Available => 1000,
        Indexed   => 0,
    }

DocumentList()

Method that returns a list of all documents for a certain DocumentType, that are either not indexed yet, or that are marked for re-indexing:

    my $DocumentList = $IndexManagementObject->DocumentList(
        DocumentType => 'Ticket',
    );

Returns:

    [ 1, 2, 3, 4 ... ]

DocumentAdd()

Method that indexes documents for a certain DocumentType and a set of document id's. Since an existing index is needed for indexing any documents, this method will check, if the related index already exists and tries to create it, if not.

    my $Success = $IndexManagementObject->DocumentAdd(
        DocumentType => 'Ticket',
        DocumentID   => [ 1, 2, 3 ], # can a single id or an array reference of id's.
    );

Returns 1 if the document indexing was successful and 0, if not.

DocumentExists()

Method that verifies, if a document is indexed on server side:

    my $Result = $IndexManagementObject->DocumentExists(
        DocumentType => 'Ticket',
        DocumentID   => [ 1, 2, 3, 4 ], # can a single id or an array reference of id's.

    );

Returns:

    {
        1 => 1,
        2 => 1,
        3 => 1,
        4 => 0,
    }

DocumentUpdate()

Method that updates documents, that are marked for re-indexing for a certain DocumentType.

    my $Success = $IndexManagementObject->DocumentUpdate(
        DocumentType => 'Ticket',
    );

Returns 1 if the document update was successful and 0, if not.

DocumentDelete()

Method that deletes documents from their related index, based on the DocumentType and related document id's.

    my $Success = $IndexManagementObject->DocumentDelete(
        DocumentType => 'Ticket',
        DocumentID   => [ 1, 2, 3 ], # can a single id or an array reference of id's.
    );

Returns 1 if the document deletion was successful and 0, if not.

_IndexDocumentSingle()

Method that adds a single document to a certain index.

    my $Success = $IndexManagementObject->_IndexDocumentSingle(
        IndexName    => 'otrs_ticket',
        DocumentType => 'Ticket',
        DocumentID   => 123,
        Body         => { TicketID = 123, Title => 'Test Title' ... },
        Pipeline     => 'PipelineName',
    );

Returns a true result in case of success, otherwise undef.

_IndexDocumentBulk()

Method that adds multiple documents to a certain index.

    my $Success = $IndexManagementObject->_IndexDocumentBulk(
        Documents => [
            {
                index   => 'otrs_ticket',
                type    => 'Ticket',
                id      => 123,
                _source => {
                    TicketID => 123,
                    Title    => 'Test Title',
                    ...
                },
            },
            ...
        ],
    );

Returns a true result in case of success, otherwise undef.

_UpdateDocumentSingle()

Method that updates a single document in a certain index.

    my $DocumentList = $IndexManagementObject->_UpdateDocumentSingle(
        IndexName    => 'otrs_ticket',
        DocumentType => 'Ticket',
        DocumentID   => 123,
        Body         => { TicketID = 123, Title => 'Test Title' ... },
    );

Returns a true result in case of success, otherwise undef.

_UpdateDocumentBulk()

Method that adds multiple documents to a certain index.

    my $Success = $IndexManagementObject->_UpdateDocumentBulk(
        Documents => [
            {
                index   => 'otrs_ticket',
                type    => 'Ticket',
                id      => 123,
                _source => {
                    TicketID => 123,
                    Title    => 'Test Title',
                    ...
                },
            },
            ...
        ],
    );

Returns a true result in case of success, otherwise undef.

_DeleteDocumentSingle()

Method that deletes a single document from a certain index.

    my $Success = $IndexManagementObject->_DeleteDocumentSingle(
        IndexName    => 'otrs_ticket',
        DocumentType => 'Ticket',
        DocumentID   => 123,
    );

Returns a true result in case of success, otherwise undef.

_DeleteDocumentBulk()

Method that deletes multiple documents from a certain index.

    my $Success = $IndexManagementObject->_DeleteDocumentBulk(
        IndexName    => 'otrs_ticket',
        DocumentType => 'Ticket',
        DocumentIDs   => [ 123, 456 ],
    );

Returns a true result in case of success, otherwise undef.