Uploaded image for project: 'StreamSets Data Collector'
  1. StreamSets Data Collector
  2. SDC-15039

Data loss when JDBC Producer doesn't match any columns

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2 (Critical with Workaround)
    • Resolution: Fixed
    • Affects Version/s: 3.16.0
    • Fix Version/s: 3.18.0
    • Component/s: None
    • Labels:
    • Testing Status:
      STF Testing Required
    • Testing Description:
      STF test was merged with this one.
    • Team:
      Data Plane
    • STF Test:

      Description

      When the JDBC Producer receives a record with no columns that match the target database table, it emits a warning in the log: "No parameters found for record with <>; skipping"

      The pipeline reports these skipped records as being successfully written.  That is data loss.  The records should instead be sent to error.

       

      The specific repro case I found was reading from an MSSQL table with lower case column names, writing to an identical schema in Oracle with upper case column names.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jarcec Jarcec
              Reporter:
              dean Dean Proctor
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: