Uploaded image for project: 'StreamSets Data Collector'
  1. StreamSets Data Collector
  2. SDC-5393

ElasticSearch Destination should optionally create ES mapping if it doesn't exist

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: P3 (Limited Impact)
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      It seems that the ES Destination currently assumes that the mapping specified either already exists or leans on the Dynamic Mapping feature to create the Mapping for the type. This causes problems for records where the detection uses an incorrect type. For instance, we have records that have the initial values that look like dates (falls under the dynamic_date_formats bucket described at https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-field-mapping.html#date-detection).

      It would be nice to have the ability for DataCollector to create the mapping explicitly from the schema from the source instead.

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            fawad Fawad Halim (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: