Uploaded image for project: 'StreamSets Data Collector'
  1. StreamSets Data Collector
  2. SDC-13307

Unique keys conversion error in MongoDB Destination

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P3 (Limited Impact)
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.17.0
    • Component/s: None
    • Labels:
    • Environment:

       
       

    • Testing Status:
      STF Testing Required
    • Team:
      Data Plane
    • STF Test:

      Description

      In the destination, unique keys are being converted to strings so updates on integer fields will not work.

      Nested fields (e.g. /orders/order_item_id will create a field in mongo called orders/order_item_id) which will cause issues.

      The issue in the code can be found on lines 150 and 165: https://github.com/streamsets/datacollector/blob/master/mongodb-protolib/src/main/java/com/streamsets/pipeline/stage/destination/mongodb/MongoDBTarget.java

       

      replaceKey.put(removeLeadingSlash(key), record.get(key).getValueAsString())

      Leading slash still needs to be removed, but other slashes need to be replaced with periods. The data type for the key should not be modified.
       

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              sebas Sebastian Sanchez
              Reporter:
              nathan Nathan Le
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: