XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2 (Critical with Workaround)
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Doc Impact:
      No
    • Testing Status:
      Not Required
    • Team:
      Data Plane

      Description

      When using SQL Parser stage and we try to parse a query as, for example, Dev Raw Data with the below value:

       

      {
       "Column1": "INSERT INTO table1 VALUES (1,'test')"
      }

       

      If we run the pipeline a NPE is thrown:

      ERROR ProductionPipelineRunnable *admin   ProductionPipelineRunnable-parserd80f0db5-7fbf-4a90-8117-0e4795bbc6d4-parser java.lang.NullPointerException at com.streamsets.pipeline.lib.jdbc.parser.sql.SQLListener.format(SQLListener.java:169) at com.streamsets.pipeline.lib.jdbc.parser.sql.SQLListener.extractTableAndSchema(SQLListener.java:61) at com.streamsets.pipeline.lib.jdbc.parser.sql.SQLListener.enterInsert_into_clause(SQLListener.java:85) at plsql.plsqlParser$Insert_into_clauseContext.enterRule(plsqlParser.java:24108) at org.antlr.v4.runtime.tree.ParseTreeWalker.enterRule(ParseTreeWalker.java:66) at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:49) at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52) at com.streamsets.pipeline.stage.processor.parser.sql.SqlParserProcessor.process(SqlParserProcessor.java:169) at com.streamsets.pipeline.stage.processor.parser.sql.SqlParserProcessor.process(SqlParserProcessor.java:152) at com.streamsets.pipeline.api.base.SingleLaneProcessor.process(SingleLaneProcessor.java:95) at com.streamsets.pipeline.api.base.configurablestage.DProcessor.process(DProcessor.java:35) at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:299) at com.streamsets.pipeline.api.impl.CreateByRef.call(CreateByRef.java:40) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:244) at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:311) at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:220) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.processPipe(ProductionPipelineRunner.java:854) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.lambda$executeRunner$3(ProductionPipelineRunner.java:898) at com.streamsets.datacollector.runner.PipeRunner.acceptConsumer(PipeRunner.java:221) at com.streamsets.datacollector.runner.PipeRunner.executeBatch(PipeRunner.java:142) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.executeRunner(ProductionPipelineRunner.java:897) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runSourceLessBatch(ProductionPipelineRunner.java:875) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runPollSource(ProductionPipelineRunner.java:599) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.run(ProductionPipelineRunner.java:390) at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:516) at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:112) at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:75) at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:718) at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:151) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226) at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:33) at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

      Pipeline attached for reference.

       

      UPDATE:

      This is failing also with the following query:
       INSERT INTO mc("Part", "Cost") VALUES('levers', 250)
      Based on our testing, it seems that the SQL Parser processor expects both schema and table names inside of double quotation marks, like:
       INSERT INTO "sdc"."mc"("Part", "Cost") VALUES('levers', 250)
       

      {{}}

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              adria.aguila Adria Aguila
              Reporter:
              mel Mel Álvarez
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: