Uploaded image for project: 'StreamSets Data Collector'
  1. StreamSets Data Collector
  2. SDC-9270

Azure Event Hub Consumer does not consume records

    Details

    • Type: Bug
    • Status: Open
    • Priority: P2 (Critical with Workaround)
    • Resolution: Unresolved
    • Affects Version/s: 3.3.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

       The following error occurs:

      2018-06-13 17:38:05,342	event-hub-sanity-test-stf/eventhubsanityteststf35926887-7727-4d3b-9f5d-0f5521f69456	WARN	host eventhubsanityteststf35926887-7727-4d3b-9f5d-0f5521f694560: Failure getting/acquiring lease, skipping	PartitionManager	*admin		pool-16-thread-5
      java.lang.RuntimeException: java.security.AccessControlException: access denied ("java.net.SocketPermission" "<storage account name>.blob.core.windows.net" "resolve") at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1455) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2979) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:489) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338) at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:76) at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:209) at com.microsoft.azure.storage.blob.CloudBlob.downloadAttributes(CloudBlob.java:1341) at com.microsoft.azure.eventprocessorhost.AzureBlobLease.lambda$isExpired$0(AzureBlobLease.java:87) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) Caused by: java.security.AccessControlException: access denied ("java.net.SocketPermission" "stfsandbox.blob.core.windows.net" "resolve") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) at java.security.AccessController.checkPermission(AccessController.java:884) at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) at java.lang.SecurityManager.checkConnect(SecurityManager.java:1048) at java.net.InetAddress.getAllByName0(InetAddress.java:1268) at java.net.InetAddress.getAllByName(InetAddress.java:1192) at java.net.InetAddress.getAllByName(InetAddress.java:1126) at java.net.InetAddress.getByName(InetAddress.java:1076) at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:372) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191) at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441) at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338) at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:115) ... 8 more
      

      We need to research if there's a bug on Azure Event Hub SDK, e.g. https://github.com/Azure/azure-event-hubs-java/issues/336 seems to be related. and if the code change is required to solve the above issue.


      Workaround - disable security policy on SDC

      export SDC_SECURITY_MANAGER_ENABLED=false 

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jisun Ji Sun Kim
              • Votes:
                3 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: