.. index:: single: language_detection_tokenizer .. _language_detection_tokenizer/0:

.. rst-class:: right

object

language_detection_tokenizer

Language-neutral tokenizer provider for language detection.

| Availability: | logtalk_load(language_detection(loader))

| Author: Paulo Moura | Version: 1:0:0 | Date: 2026-09-06

| Compilation flags: | static, context_switching_calls

| Implements: | public :ref:`tokenizer_language_protocol <tokenizer_language_protocol/0>` | Imports: | public :ref:`tokenizer_rules <tokenizer_rules/0>`

| Remarks: | (none)

| Inherited public predicates: | Â :ref:`options_protocol/0::check_option/1` Â :ref:`options_protocol/0::check_options/1` Â :ref:`options_protocol/0::default_option/1` Â :ref:`options_protocol/0::default_options/1` Â :ref:`options_protocol/0::option/2` Â :ref:`options_protocol/0::option/3` Â :ref:`tokenizer_language_protocol/0::split_sentence_codes/3` Â :ref:`tokenizer_language_protocol/0::tokenize_codes/3` Â :ref:`options_protocol/0::valid_option/1` Â :ref:`options_protocol/0::valid_options/1` Â

.. contents:: :local: :backlinks: top

Public predicates

(no local declarations; see entity ancestors if any)

Protected predicates

(no local declarations; see entity ancestors if any)

Private predicates

(no local declarations; see entity ancestors if any)

Operators

(none)

.. seealso::

:ref:`tokenizer(Representation,Language) <tokenizer/2>`, :ref:`tokenizer_rules <tokenizer_rules/0>`