java - Spark Dynamic Allocation in Standalone Cluster Failing My Application -

i running spark 1.5.2 build on scala 2.11 on windows 7 cluster on standalone mode , two spark applications cores allocated both application.

dynamic allocation enabled configuration

when run 2 instances of application none of application completes.

here application -

package com.cleartrail.clearinsight.spark;  import java.util.arrays; import java.util.list;  import org.apache.spark.sparkconf; import org.apache.spark.sparkcontext; import org.apache.spark.api.java.javapairrdd; import org.apache.spark.api.java.javardd; import org.apache.spark.api.java.javasparkcontext;  public class main {      public static void main(string[] args) {          sparkconf conf = new sparkconf().setappname("sparkpoc").setmaster("spark://172.50.33.159:7077").set("spark.dynamicallocation.enabled", "true")                 .set("spark.shuffle.service.enabled", "true").set("spark.dynamicallocation.executoridletimeout", "10s");// .set("spark.cores.max", "2");         // sparkconf conf = new sparkconf().setappname("sparkpoc").setmaster("local");          sparkcontext sparkcontext = sparkcontext.getorcreate(conf);         javasparkcontext sc = new javasparkcontext(sparkcontext);         sc.addjar("./target/spark-poc-0.0.1-snapshot.jar");         try {             list<integer> data = arrays.aslist(1, 2, 3, 4, 5);             javardd<integer> distdata = sc.parallelize(data);             system.out.println(distdata.count());         }         {             sc.close();         }     } } 

errors on console -

using spark's default log4j profile: org/apache/spark/log4j-defaults.properties 16/06/10 14:20:03 info sparkcontext: running spark version 1.5.2 16/06/10 14:20:03 warn nativecodeloader: unable load native-hadoop library platform... using builtin-java classes applicable 16/06/10 14:20:03 info securitymanager: changing view acls to: pranjal.jaju 16/06/10 14:20:03 info securitymanager: changing modify acls to: pranjal.jaju 16/06/10 14:20:03 info securitymanager: securitymanager: authentication disabled; ui acls disabled; users view permissions: set(pranjal.jaju); users modify permissions: set(pranjal.jaju) 16/06/10 14:20:04 info slf4jlogger: slf4jlogger started 16/06/10 14:20:04 info remoting: starting remoting 16/06/10 14:20:04 info remoting: remoting started; listening on addresses :[akka.tcp://sparkdriver@172.50.33.159:58257] 16/06/10 14:20:04 info utils: started service 'sparkdriver' on port 58257. 16/06/10 14:20:04 info sparkenv: registering mapoutputtracker 16/06/10 14:20:04 info sparkenv: registering blockmanagermaster 16/06/10 14:20:04 info diskblockmanager: created local directory @ c:\users\pranjal.jaju\appdata\local\temp\blockmgr-99f1ab11-9aff-40f3-b2b5-3c992972657c 16/06/10 14:20:04 info memorystore: memorystore started capacity 965.8 mb 16/06/10 14:20:04 info httpfileserver: http file server directory c:\users\pranjal.jaju\appdata\local\temp\spark-bcb3998e-0cd5-4146-9a01-ec237325fc5f\httpd-ffd9b538-3ebb-47ef-9005-4c66383466b0 16/06/10 14:20:04 info httpserver: starting http server 16/06/10 14:20:04 info utils: started service 'http file server' on port 58259. 16/06/10 14:20:05 info sparkenv: registering outputcommitcoordinator 16/06/10 14:20:05 info utils: started service 'sparkui' on port 4040. 16/06/10 14:20:05 info sparkui: started sparkui @ http://172.50.33.159:4040 16/06/10 14:20:05 warn metricssystem: using default name dagscheduler source because spark.app.id not set. 16/06/10 14:20:05 info appclient$clientendpoint: connecting master spark://172.50.33.159:7077... 16/06/10 14:20:05 info sparkdeployschedulerbackend: connected spark cluster app id app-20160610142005-0004 16/06/10 14:20:05 info appclient$clientendpoint: executor added: app-20160610142005-0004/0 on worker-20160610141809-172.50.33.159-57648 (172.50.33.159:57648) 4 cores 16/06/10 14:20:05 info sparkdeployschedulerbackend: granted executor id app-20160610142005-0004/0 on hostport 172.50.33.159:57648 4 cores, 1024.0 mb ram 16/06/10 14:20:05 info appclient$clientendpoint: executor added: app-20160610142005-0004/1 on worker-20160610141326-172.50.33.159-56922 (172.50.33.159:56922) 4 cores 16/06/10 14:20:05 info sparkdeployschedulerbackend: granted executor id app-20160610142005-0004/1 on hostport 172.50.33.159:56922 4 cores, 1024.0 mb ram 16/06/10 14:20:05 info appclient$clientendpoint: executor updated: app-20160610142005-0004/0 loading 16/06/10 14:20:05 info appclient$clientendpoint: executor updated: app-20160610142005-0004/0 running 16/06/10 14:20:05 info appclient$clientendpoint: executor updated: app-20160610142005-0004/1 running 16/06/10 14:20:05 info appclient$clientendpoint: executor updated: app-20160610142005-0004/1 loading 16/06/10 14:20:06 info utils: started service 'org.apache.spark.network.netty.nettyblocktransferservice' on port 58279. 16/06/10 14:20:06 info nettyblocktransferservice: server created on 58279 16/06/10 14:20:06 info blockmanagermaster: trying register blockmanager 16/06/10 14:20:06 info blockmanagermasterendpoint: registering block manager 172.50.33.159:58279 965.8 mb ram, blockmanagerid(driver, 172.50.33.159, 58279) 16/06/10 14:20:06 info blockmanagermaster: registered blockmanager 16/06/10 14:20:06 info sparkdeployschedulerbackend: schedulerbackend ready scheduling beginning after reached minregisteredresourcesratio: 0.0 16/06/10 14:20:07 info sparkcontext: added jar ./target/spark-poc-0.0.1-snapshot.jar @ http://172.50.33.159:58259/jars/spark-poc-0.0.1-snapshot.jar timestamp 1465548607657 16/06/10 14:20:11 info sparkdeployschedulerbackend: registered executor: akkarpcendpointref(actor[akka.tcp://sparkexecutor@172.50.33.159:58341/user/executor#-804456045]) id 1 16/06/10 14:20:11 info executorallocationmanager: new executor 1 has registered (new total 1) 16/06/10 14:20:11 info sparkdeployschedulerbackend: registered executor: akkarpcendpointref(actor[akka.tcp://sparkexecutor@172.50.33.159:58340/user/executor#-859540224]) id 0 16/06/10 14:20:11 info executorallocationmanager: new executor 0 has registered (new total 2) 16/06/10 14:20:12 info blockmanagermasterendpoint: registering block manager 172.50.33.159:58379 530.0 mb ram, blockmanagerid(1, 172.50.33.159, 58379) 16/06/10 14:20:12 info blockmanagermasterendpoint: registering block manager 172.50.33.159:58390 530.0 mb ram, blockmanagerid(0, 172.50.33.159, 58390) 16/06/10 14:20:21 info sparkdeployschedulerbackend: requesting kill executor(s) 1 16/06/10 14:20:21 info executorallocationmanager: removing executor 1 because has been idle 10 seconds (new desired total 1) 16/06/10 14:20:21 error taskschedulerimpl: lost executor 1 on 172.50.33.159: remote rpc client disassociated 16/06/10 14:20:21 warn reliabledeliverysupervisor: association remote system [akka.tcp://sparkexecutor@172.50.33.159:58341] has failed, address gated [5000] ms. reason: [disassociated]  16/06/10 14:20:21 info executorallocationmanager: existing executor 1 has been removed (new total 1) 16/06/10 14:20:21 info dagscheduler: executor lost: 1 (epoch 0) 16/06/10 14:20:21 info blockmanagermasterendpoint: trying remove executor 1 blockmanagermaster. 16/06/10 14:20:21 info blockmanagermasterendpoint: removing block manager blockmanagerid(1, 172.50.33.159, 58379) 16/06/10 14:20:21 info blockmanagermaster: removed 1 in removeexecutor 16/06/10 14:20:21 info sparkdeployschedulerbackend: requesting kill executor(s) 0 16/06/10 14:20:21 info executorallocationmanager: removing executor 0 because has been idle 10 seconds (new desired total 0) 16/06/10 14:20:21 error taskschedulerimpl: lost executor 0 on 172.50.33.159: remote rpc client disassociated 16/06/10 14:20:21 warn reliabledeliverysupervisor: association remote system [akka.tcp://sparkexecutor@172.50.33.159:58340] has failed, address gated [5000] ms. reason: [disassociated]  16/06/10 14:20:21 info executorallocationmanager: existing executor 0 has been removed (new total 0) 16/06/10 14:20:21 info dagscheduler: executor lost: 0 (epoch 0) 16/06/10 14:20:21 info blockmanagermasterendpoint: trying remove executor 0 blockmanagermaster. 16/06/10 14:20:21 info blockmanagermasterendpoint: removing block manager blockmanagerid(0, 172.50.33.159, 58390) 16/06/10 14:20:21 info blockmanagermaster: removed 0 in removeexecutor 16/06/10 14:20:30 info sparkcontext: starting job: count @ main.java:26 16/06/10 14:20:30 info dagscheduler: got job 0 (count @ main.java:26) 2 output partitions 16/06/10 14:20:30 info dagscheduler: final stage: resultstage 0(count @ main.java:26) 16/06/10 14:20:30 info dagscheduler: parents of final stage: list() 16/06/10 14:20:30 info dagscheduler: missing parents: list() 16/06/10 14:20:30 info dagscheduler: submitting resultstage 0 (parallelcollectionrdd[0] @ parallelize @ main.java:25), has no missing parents 16/06/10 14:20:31 info memorystore: ensurefreespace(1384) called curmem=0, maxmem=1012704215 16/06/10 14:20:31 info memorystore: block broadcast_0 stored values in memory (estimated size 1384.0 b, free 965.8 mb) 16/06/10 14:20:31 info memorystore: ensurefreespace(942) called curmem=1384, maxmem=1012704215 16/06/10 14:20:31 info memorystore: block broadcast_0_piece0 stored bytes in memory (estimated size 942.0 b, free 965.8 mb) 16/06/10 14:20:31 info blockmanagerinfo: added broadcast_0_piece0 in memory on 172.50.33.159:58279 (size: 942.0 b, free: 965.8 mb) 16/06/10 14:20:31 info sparkcontext: created broadcast 0 broadcast @ dagscheduler.scala:861 16/06/10 14:20:31 info dagscheduler: submitting 2 missing tasks resultstage 0 (parallelcollectionrdd[0] @ parallelize @ main.java:25) 16/06/10 14:20:31 info taskschedulerimpl: adding task set 0.0 2 tasks 16/06/10 14:20:32 info appclient$clientendpoint: executor added: app-20160610142005-0004/2 on worker-20160610141809-172.50.33.159-57648 (172.50.33.159:57648) 4 cores 16/06/10 14:20:32 info executorallocationmanager: requesting 1 new executor because tasks backlogged (new desired total 1) 16/06/10 14:20:32 info sparkdeployschedulerbackend: granted executor id app-20160610142005-0004/2 on hostport 172.50.33.159:57648 4 cores, 1024.0 mb ram 16/06/10 14:20:32 info appclient$clientendpoint: executor updated: app-20160610142005-0004/2 loading 16/06/10 14:20:32 info appclient$clientendpoint: executor updated: app-20160610142005-0004/2 running 16/06/10 14:20:33 info appclient$clientendpoint: executor added: app-20160610142005-0004/3 on worker-20160610141326-172.50.33.159-56922 (172.50.33.159:56922) 3 cores 16/06/10 14:20:33 info sparkdeployschedulerbackend: granted executor id app-20160610142005-0004/3 on hostport 172.50.33.159:56922 3 cores, 1024.0 mb ram 16/06/10 14:20:33 info executorallocationmanager: requesting 1 new executor because tasks backlogged (new desired total 2) 16/06/10 14:20:33 info appclient$clientendpoint: executor updated: app-20160610142005-0004/3 running 16/06/10 14:20:33 info appclient$clientendpoint: executor updated: app-20160610142005-0004/3 loading 16/06/10 14:20:36 info sparkdeployschedulerbackend: registered executor: akkarpcendpointref(actor[akka.tcp://sparkexecutor@172.50.33.159:58510/user/executor#1976924864]) id 2 16/06/10 14:20:36 info executorallocationmanager: new executor 2 has registered (new total 1) 16/06/10 14:20:36 info tasksetmanager: starting task 0.0 in stage 0.0 (tid 0, 172.50.33.159, process_local, 2136 bytes) 16/06/10 14:20:36 info tasksetmanager: starting task 1.0 in stage 0.0 (tid 1, 172.50.33.159, process_local, 2146 bytes) 16/06/10 14:20:36 info blockmanagermasterendpoint: registering block manager 172.50.33.159:58543 530.0 mb ram, blockmanagerid(2, 172.50.33.159, 58543) 16/06/10 14:20:37 info sparkdeployschedulerbackend: registered executor: akkarpcendpointref(actor[akka.tcp://sparkexecutor@172.50.33.159:58565/user/executor#540501534]) id 3 16/06/10 14:20:37 info executorallocationmanager: new executor 3 has registered (new total 2) 16/06/10 14:20:37 info blockmanagermasterendpoint: registering block manager 172.50.33.159:58581 530.0 mb ram, blockmanagerid(3, 172.50.33.159, 58581) 16/06/10 14:20:47 info sparkdeployschedulerbackend: requesting kill executor(s) 3 16/06/10 14:20:47 info executorallocationmanager: removing executor 3 because has been idle 10 seconds (new desired total 1) 16/06/10 14:20:47 error taskschedulerimpl: lost executor 3 on 172.50.33.159: remote rpc client disassociated 16/06/10 14:20:47 warn reliabledeliverysupervisor: association remote system [akka.tcp://sparkexecutor@172.50.33.159:58565] has failed, address gated [5000] ms. reason: [disassociated]  16/06/10 14:20:47 info tasksetmanager: re-queueing tasks 3 taskset 0.0 16/06/10 14:20:47 info dagscheduler: executor lost: 3 (epoch 0) 16/06/10 14:20:47 info blockmanagermasterendpoint: trying remove executor 3 blockmanagermaster. 16/06/10 14:20:47 info blockmanagermasterendpoint: removing block manager blockmanagerid(3, 172.50.33.159, 58581) 16/06/10 14:20:47 info blockmanagermaster: removed 3 in removeexecutor 16/06/10 14:20:47 info executorallocationmanager: existing executor 3 has been removed (new total 1) 16/06/10 14:20:50 error taskschedulerimpl: lost executor 2 on 172.50.33.159: remote rpc client disassociated 16/06/10 14:20:50 info tasksetmanager: re-queueing tasks 2 taskset 0.0 16/06/10 14:20:50 warn reliabledeliverysupervisor: association remote system [akka.tcp://sparkexecutor@172.50.33.159:58510] has failed, address gated [5000] ms. reason: [disassociated]  16/06/10 14:20:50 warn tasksetmanager: lost task 1.0 in stage 0.0 (tid 1, 172.50.33.159): executorlostfailure (executor 2 lost) 16/06/10 14:20:50 warn tasksetmanager: lost task 0.0 in stage 0.0 (tid 0, 172.50.33.159): executorlostfailure (executor 2 lost) 16/06/10 14:20:50 info dagscheduler: executor lost: 2 (epoch 0) 16/06/10 14:20:50 info blockmanagermasterendpoint: trying remove executor 2 blockmanagermaster. 16/06/10 14:20:50 info blockmanagermasterendpoint: removing block manager blockmanagerid(2, 172.50.33.159, 58543) 16/06/10 14:20:50 info blockmanagermaster: removed 2 in removeexecutor 16/06/10 14:20:50 info executorallocationmanager: existing executor 2 has been removed (new total 0) 16/06/10 14:20:50 info appclient$clientendpoint: executor updated: app-20160610142005-0004/2 exited (command exited code 1) 16/06/10 14:20:50 info sparkdeployschedulerbackend: executor app-20160610142005-0004/2 removed: command exited code 1 16/06/10 14:20:50 info sparkdeployschedulerbackend: asked remove non-existent executor 2 16/06/10 14:20:50 info appclient$clientendpoint: executor added: app-20160610142005-0004/4 on worker-20160610141809-172.50.33.159-57648 (172.50.33.159:57648) 4 cores 16/06/10 14:20:50 info sparkdeployschedulerbackend: granted executor id app-20160610142005-0004/4 on hostport 172.50.33.159:57648 4 cores, 1024.0 mb ram 16/06/10 14:20:50 info appclient$clientendpoint: executor updated: app-20160610142005-0004/4 loading 16/06/10 14:20:50 info appclient$clientendpoint: executor updated: app-20160610142005-0004/4 running 16/06/10 14:20:53 info sparkdeployschedulerbackend: registered executor: akkarpcendpointref(actor[akka.tcp://sparkexecutor@172.50.33.159:58657/user/executor#-1698957872]) id 4 16/06/10 14:20:53 info tasksetmanager: starting task 0.1 in stage 0.0 (tid 2, 172.50.33.159, process_local, 2136 bytes) 16/06/10 14:20:53 info tasksetmanager: starting task 1.1 in stage 0.0 (tid 3, 172.50.33.159, process_local, 2146 bytes) 16/06/10 14:20:53 info executorallocationmanager: new executor 4 has registered (new total 1) 16/06/10 14:20:53 info blockmanagermasterendpoint: registering block manager 172.50.33.159:58677 530.0 mb ram, blockmanagerid(4, 172.50.33.159, 58677) 16/06/10 14:21:07 error taskschedulerimpl: lost executor 4 on 172.50.33.159: remote rpc client disassociated 16/06/10 14:21:07 info tasksetmanager: re-queueing tasks 4 taskset 0.0 16/06/10 14:21:07 warn tasksetmanager: lost task 0.1 in stage 0.0 (tid 2, 172.50.33.159): executorlostfailure (executor 4 lost) 16/06/10 14:21:07 warn tasksetmanager: lost task 1.1 in stage 0.0 (tid 3, 172.50.33.159): executorlostfailure (executor 4 lost) 16/06/10 14:21:07 info dagscheduler: executor lost: 4 (epoch 0) 16/06/10 14:21:07 info blockmanagermasterendpoint: trying remove executor 4 blockmanagermaster. 16/06/10 14:21:07 info blockmanagermasterendpoint: removing block manager blockmanagerid(4, 172.50.33.159, 58677) 16/06/10 14:21:07 info blockmanagermaster: removed 4 in removeexecutor 16/06/10 14:21:07 warn reliabledeliverysupervisor: association remote system [akka.tcp://sparkexecutor@172.50.33.159:58657] has failed, address gated [5000] ms. reason: [disassociated]  16/06/10 14:21:07 info executorallocationmanager: existing executor 4 has been removed (new total 0) 16/06/10 14:21:07 info appclient$clientendpoint: executor updated: app-20160610142005-0004/4 exited (command exited code 1) 16/06/10 14:21:07 info sparkdeployschedulerbackend: executor app-20160610142005-0004/4 removed: command exited code 1 16/06/10 14:21:07 info sparkdeployschedulerbackend: asked remove non-existent executor 4 16/06/10 14:21:07 info appclient$clientendpoint: executor added: app-20160610142005-0004/5 on worker-20160610141809-172.50.33.159-57648 (172.50.33.159:57648) 4 cores 16/06/10 14:21:07 info sparkdeployschedulerbackend: granted executor id app-20160610142005-0004/5 on hostport 172.50.33.159:57648 4 cores, 1024.0 mb ram 16/06/10 14:21:07 info appclient$clientendpoint: executor updated: app-20160610142005-0004/5 loading 16/06/10 14:21:07 info appclient$clientendpoint: executor updated: app-20160610142005-0004/5 running 16/06/10 14:21:10 info sparkdeployschedulerbackend: registered executor: akkarpcendpointref(actor[akka.tcp://sparkexecutor@172.50.33.159:58754/user/executor#-457099261]) id 5 16/06/10 14:21:10 info tasksetmanager: starting task 1.2 in stage 0.0 (tid 4, 172.50.33.159, process_local, 2146 bytes) 16/06/10 14:21:10 info tasksetmanager: starting task 0.2 in stage 0.0 (tid 5, 172.50.33.159, process_local, 2136 bytes) 16/06/10 14:21:10 info executorallocationmanager: new executor 5 has registered (new total 1) 16/06/10 14:21:10 info blockmanagermasterendpoint: registering block manager 172.50.33.159:58773 530.0 mb ram, blockmanagerid(5, 172.50.33.159, 58773) 16/06/10 14:21:23 error taskschedulerimpl: lost executor 5 on 172.50.33.159: remote rpc client disassociated 16/06/10 14:21:23 info tasksetmanager: re-queueing tasks 5 taskset 0.0 16/06/10 14:21:23 warn tasksetmanager: lost task 0.2 in stage 0.0 (tid 5, 172.50.33.159): executorlostfailure (executor 5 lost) 16/06/10 14:21:23 warn tasksetmanager: lost task 1.2 in stage 0.0 (tid 4, 172.50.33.159): executorlostfailure (executor 5 lost) 16/06/10 14:21:23 info dagscheduler: executor lost: 5 (epoch 0) 16/06/10 14:21:23 info blockmanagermasterendpoint: trying remove executor 5 blockmanagermaster. 16/06/10 14:21:23 info blockmanagermasterendpoint: removing block manager blockmanagerid(5, 172.50.33.159, 58773) 16/06/10 14:21:23 info blockmanagermaster: removed 5 in removeexecutor 16/06/10 14:21:23 warn reliabledeliverysupervisor: association remote system [akka.tcp://sparkexecutor@172.50.33.159:58754] has failed, address gated [5000] ms. reason: [disassociated]  16/06/10 14:21:23 info executorallocationmanager: existing executor 5 has been removed (new total 0) 16/06/10 14:21:23 warn executorallocationmanager: attempted mark unknown executor 5 idle 16/06/10 14:21:24 info appclient$clientendpoint: executor updated: app-20160610142005-0004/5 exited (command exited code 1) 16/06/10 14:21:24 info sparkdeployschedulerbackend: executor app-20160610142005-0004/5 removed: command exited code 1 16/06/10 14:21:24 info sparkdeployschedulerbackend: asked remove non-existent executor 5 16/06/10 14:21:24 info appclient$clientendpoint: executor added: app-20160610142005-0004/6 on worker-20160610141809-172.50.33.159-57648 (172.50.33.159:57648) 4 cores 16/06/10 14:21:24 info sparkdeployschedulerbackend: granted executor id app-20160610142005-0004/6 on hostport 172.50.33.159:57648 4 cores, 1024.0 mb ram 16/06/10 14:21:24 info appclient$clientendpoint: executor updated: app-20160610142005-0004/6 running 16/06/10 14:21:24 info appclient$clientendpoint: executor updated: app-20160610142005-0004/6 loading 16/06/10 14:21:26 info sparkdeployschedulerbackend: registered executor: akkarpcendpointref(actor[akka.tcp://sparkexecutor@172.50.33.159:58853/user/executor#-1050109075]) id 6 16/06/10 14:21:26 info tasksetmanager: starting task 1.3 in stage 0.0 (tid 6, 172.50.33.159, process_local, 2146 bytes) 16/06/10 14:21:26 info tasksetmanager: starting task 0.3 in stage 0.0 (tid 7, 172.50.33.159, process_local, 2136 bytes) 16/06/10 14:21:26 info executorallocationmanager: new executor 6 has registered (new total 1) 16/06/10 14:21:26 info blockmanagermasterendpoint: registering block manager 172.50.33.159:58872 530.0 mb ram, blockmanagerid(6, 172.50.33.159, 58872) 16/06/10 14:21:40 error taskschedulerimpl: lost executor 6 on 172.50.33.159: remote rpc client disassociated 16/06/10 14:21:40 info tasksetmanager: re-queueing tasks 6 taskset 0.0 16/06/10 14:21:40 warn reliabledeliverysupervisor: association remote system [akka.tcp://sparkexecutor@172.50.33.159:58853] has failed, address gated [5000] ms. reason: [disassociated]  16/06/10 14:21:40 warn tasksetmanager: lost task 0.3 in stage 0.0 (tid 7, 172.50.33.159): executorlostfailure (executor 6 lost) 16/06/10 14:21:40 error tasksetmanager: task 0 in stage 0.0 failed 4 times; aborting job 16/06/10 14:21:40 warn tasksetmanager: lost task 1.3 in stage 0.0 (tid 6, 172.50.33.159): executorlostfailure (executor 6 lost) 16/06/10 14:21:40 info taskschedulerimpl: removed taskset 0.0, tasks have completed, pool  16/06/10 14:21:40 info executorallocationmanager: existing executor 6 has been removed (new total 0) 16/06/10 14:21:40 info taskschedulerimpl: cancelling stage 0 16/06/10 14:21:40 info dagscheduler: resultstage 0 (count @ main.java:26) failed in 69.342 s 16/06/10 14:21:40 info dagscheduler: job 0 failed: count @ main.java:26, took 69.603206 s 16/06/10 14:21:40 warn executorallocationmanager: no stages running, numrunningtasks != 0 16/06/10 14:21:40 info dagscheduler: executor lost: 6 (epoch 0) 16/06/10 14:21:40 warn executorallocationmanager: attempted mark unknown executor 6 idle 16/06/10 14:21:40 info blockmanagermasterendpoint: trying remove executor 6 blockmanagermaster. 16/06/10 14:21:40 info blockmanagermasterendpoint: removing block manager blockmanagerid(6, 172.50.33.159, 58872) 16/06/10 14:21:40 info blockmanagermaster: removed 6 in removeexecutor 16/06/10 14:21:40 info appclient$clientendpoint: executor updated: app-20160610142005-0004/6 exited (command exited code 1) 16/06/10 14:21:40 info sparkdeployschedulerbackend: executor app-20160610142005-0004/6 removed: command exited code 1 16/06/10 14:21:40 info sparkdeployschedulerbackend: asked remove non-existent executor 6 16/06/10 14:21:40 info appclient$clientendpoint: executor added: app-20160610142005-0004/7 on worker-20160610141809-172.50.33.159-57648 (172.50.33.159:57648) 4 cores 16/06/10 14:21:40 info sparkdeployschedulerbackend: granted executor id app-20160610142005-0004/7 on hostport 172.50.33.159:57648 4 cores, 1024.0 mb ram 16/06/10 14:21:40 info appclient$clientendpoint: executor updated: app-20160610142005-0004/7 loading 16/06/10 14:21:40 info appclient$clientendpoint: executor updated: app-20160610142005-0004/7 running 16/06/10 14:21:43 info sparkdeployschedulerbackend: registered executor: akkarpcendpointref(actor[akka.tcp://sparkexecutor@172.50.33.159:58949/user/executor#645252204]) id 7 16/06/10 14:21:43 info executorallocationmanager: new executor 7 has registered (new total 1) 16/06/10 14:21:43 info blockmanagermasterendpoint: registering block manager 172.50.33.159:58968 530.0 mb ram, blockmanagerid(7, 172.50.33.159, 58968) 16/06/10 14:21:48 info sparkui: stopped spark web ui @ http://172.50.33.159:4040 16/06/10 14:21:48 info dagscheduler: stopping dagscheduler 16/06/10 14:21:48 info sparkdeployschedulerbackend: shutting down executors 16/06/10 14:21:48 info sparkdeployschedulerbackend: asking each executor shut down 16/06/10 14:21:48 warn reliabledeliverysupervisor: association remote system [akka.tcp://sparkexecutor@172.50.33.159:58949] has failed, address gated [5000] ms. reason: [disassociated]  16/06/10 14:21:48 info mapoutputtrackermasterendpoint: mapoutputtrackermasterendpoint stopped! 16/06/10 14:21:48 info memorystore: memorystore cleared 16/06/10 14:21:48 info blockmanager: blockmanager stopped 16/06/10 14:21:48 info blockmanagermaster: blockmanagermaster stopped 16/06/10 14:21:48 info outputcommitcoordinator$outputcommitcoordinatorendpoint: outputcommitcoordinator stopped! 16/06/10 14:21:48 info sparkcontext: stopped sparkcontext 16/06/10 14:21:48 info remoteactorrefprovider$remotingterminator: shutting down remote daemon. 16/06/10 14:21:48 info remoteactorrefprovider$remotingterminator: remote daemon shut down; proceeding flushing remote transports. 16/06/10 14:21:48 info remoteactorrefprovider$remotingterminator: remoting shut down. exception in thread "main" org.apache.spark.sparkexception: job aborted due stage failure: task 0 in stage 0.0 failed 4 times, recent failure: lost task 0.3 in stage 0.0 (tid 7, 172.50.33.159): executorlostfailure (executor 6 lost) driver stacktrace:     @ org.apache.spark.scheduler.dagscheduler.org$apache$spark$scheduler$dagscheduler$$failjobandindependentstages(dagscheduler.scala:1283)     @ org.apache.spark.scheduler.dagscheduler$$anonfun$abortstage$1.apply(dagscheduler.scala:1271)     @ org.apache.spark.scheduler.dagscheduler$$anonfun$abortstage$1.apply(dagscheduler.scala:1270)     @ scala.collection.mutable.resizablearray$class.foreach(resizablearray.scala:59)     @ scala.collection.mutable.arraybuffer.foreach(arraybuffer.scala:48)     @ org.apache.spark.scheduler.dagscheduler.abortstage(dagscheduler.scala:1270)     @ org.apache.spark.scheduler.dagscheduler$$anonfun$handletasksetfailed$1.apply(dagscheduler.scala:697)     @ org.apache.spark.scheduler.dagscheduler$$anonfun$handletasksetfailed$1.apply(dagscheduler.scala:697)     @ scala.option.foreach(option.scala:257)     @ org.apache.spark.scheduler.dagscheduler.handletasksetfailed(dagscheduler.scala:697)     @ org.apache.spark.scheduler.dagschedulereventprocessloop.doonreceive(dagscheduler.scala:1496)     @ org.apache.spark.scheduler.dagschedulereventprocessloop.onreceive(dagscheduler.scala:1458)     @ org.apache.spark.scheduler.dagschedulereventprocessloop.onreceive(dagscheduler.scala:1447)     @ org.apache.spark.util.eventloop$$anon$1.run(eventloop.scala:48)     @ org.apache.spark.scheduler.dagscheduler.runjob(dagscheduler.scala:567)     @ org.apache.spark.sparkcontext.runjob(sparkcontext.scala:1824)     @ org.apache.spark.sparkcontext.runjob(sparkcontext.scala:1837)     @ org.apache.spark.sparkcontext.runjob(sparkcontext.scala:1850)     @ org.apache.spark.sparkcontext.runjob(sparkcontext.scala:1921)     @ org.apache.spark.rdd.rdd.count(rdd.scala:1125)     @ org.apache.spark.api.java.javarddlike$class.count(javarddlike.scala:445)     @ org.apache.spark.api.java.abstractjavarddlike.count(javarddlike.scala:47)     @ com.cleartrail.clearinsight.spark.main.main(main.java:26) 16/06/10 14:21:49 info shutdownhookmanager: shutdown hook called 16/06/10 14:21:49 info shutdownhookmanager: deleting directory c:\users\pranjal.jaju\appdata\local\temp\spark-bcb3998e-0cd5-4146-9a01-ec237325fc5f 

there physical shuffle service need started might did not started service in spark