Uploaded image for project: 'StreamSets Data Collector'
  1. StreamSets Data Collector
  2. SDC-13391

SQL Server CDC generates lots of trace at ERROR level.

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: P2 (Critical with Workaround)
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Doc Impact:
      No
    • Testing Status:
      STF Testing Required
    • Team:
      Data Plane
    • Stage:
      SQL Server CDC Client origin

      Description

      Related to ESC-691.

      This trace seems to be generated for every request to SQL Server.  In the customer's case with Queries per second set to 10, it's hitting the log about 500-550 times per minute.  In my test case, there were 4 records in the CDC table, they were correctly transferred. 

       

      2020-01-31 11:36:38,787 [user:*admin] [pipeline:SQL Server CDC - trash 3x/SQLServerCDCtrash3x99d15edc-fcd7-4853-9331-3cae4bf3f8fa] [runner:] [thread:Table Jdbc Runner - 0] [stage:SQLServerCDCClient_01] ERROR AbstractTableJdbcSource - SQLException attempting to update max offsets for TableContext TableContext{schema='cdc', tableName='dbo_bob_CT'}: The multi-part identifier "__$sdc.txn_window" could not be bound.
      com.microsoft.sqlserver.jdbc.SQLServerException: The multi-part identifier "__$sdc.txn_window" could not be bound.
              at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)
              at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1621)
              at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:868)
              at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:768)
              at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194)
              at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2935)
              at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:248)
              at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:223)
              at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(SQLServerStatement.java:693)
              at com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:111)
              at com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
              at com.streamsets.pipeline.lib.jdbc.JdbcUtil.getMinMaxOffsetValueHelper(JdbcUtil.java:413)
              at com.streamsets.pipeline.lib.jdbc.JdbcUtil.getMaximumOffsetValues(JdbcUtil.java:375)
              at com.streamsets.pipeline.stage.origin.jdbc.AbstractTableJdbcSource.updateMaxOffsetsForTable(AbstractTableJdbcSource.java:574)
              at com.streamsets.pipeline.lib.jdbc.multithread.MultithreadedTableProvider.removePartitionIfNeeded(MultithreadedTableProvider.java:553)
              at com.streamsets.pipeline.lib.jdbc.multithread.MultithreadedTableProvider.reportDataOrNoMoreData(MultithreadedTableProvider.java:750)
              at com.streamsets.pipeline.lib.jdbc.multithread.JdbcBaseRunnable.generateBatchAndCommitOffset(JdbcBaseRunnable.java:292)
              at com.streamsets.pipeline.lib.jdbc.multithread.JdbcBaseRunnable.run(JdbcBaseRunnable.java:178)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      

      This is just the top of the stack, check the attached log file for the entire output from my test case. 

      Also, the workaround, for the time being, is to add `log4j.logger.com.streamsets.pipeline.stage.origin.jdbc.AbstractTableJdbcSource=FATAL` to the log configs .  

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            bob bob plotts
            Votes:
            2 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated: