[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xmlblaster] server hangs when changing the size maximum of the DispathcWorkerPool

To: xmlblaster at server.xmlBlaster.org
Subject: Re: [xmlblaster] server hangs when changing the size maximum of the DispathcWorkerPool
From: Marcel Ruff <mr at marcelruff.info>
Date: Fri, 28 Sep 2007 09:10:03 +0200
In-reply-to: <0329C0D74DD1D64F98E6C6B43B5C781003289D1C at frlulms01.iv.local>
References: 46FA3C73.3040605 at marcelruff.info <20070926124615.af31f4b0 at mail.openroadsconsulting.com> <0329C0D74DD1D64F98E6C6B43B5C7810031F9EDF at frlulms01.iv.local> <46FA906C.9050601 at marcelruff.info> <0329C0D74DD1D64F98E6C6B43B5C781003289D1C at frlulms01.iv.local>
Reply-to: xmlblaster at server.xmlBlaster.org
Sender: owner-xmlblaster at server.xmlBlaster.org
User-agent: Thunderbird 2.0.0.0 (X11/20070326)

Hi Xavier,

on a first glance i can't see the reason why the limited Pool has this impact, please increase the pool size to the old value until i have discussed the issue with Michele,

thanks
Marcel

Xavier Roques wrote:

Hi Marcel,
I'm using your last fix in order to avoid dead lock.
But I encountered a new problem.
I changed my server configuration in order to have less threads.
So I added the following lines: dispatch/callback/maximumPoolSize=10 dispatch/callback/minimumPoolSize=5
But therefore I was adding some clients to my server, suddenly, I was
not more able to connect to my server.
It seems to be hung.
Below it the dump of the threads (Unfortunately the Dispathworker 8 and
9 are missing )
Thanks in advance for your answer,
Xavier. "XmlBlaster.DispatchWorkerPool.xmlBlaster_10_1_13_13_3412-7" daemon prio=10 tid=0x010fb400 nid=0xbb3 waiting for monitor entry [0xd308f000..0xd308fb70] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.util.dispatch.DispatchWorkerPool.execute(DispatchWorkerPo ol.java:103) - waiting to lock <0xdae67640> (a org.xmlBlaster.util.dispatch.DispatchWorkerPool) at org.xmlBlaster.util.dispatch.DispatchManager.startWorkerThread(DispatchM anager.java:722) - locked <0xdf5776b0> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchManager.activateDispatchWorker(Disp atchManager.java:674) at org.xmlBlaster.util.dispatch.DispatchManager.setDispatchWorkerIsActive(D ispatchManager.java:888) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:150) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Thread.java:619) "XmlBlaster.DispatchWorkerPool.xmlBlaster_10_1_13_13_3412-6" daemon prio=10 tid=0x0015a000 nid=0x199 waiting for monitor entry [0xd49ff000..0xd49ff870] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.util.dispatch.DispatchWorkerPool.execute(DispatchWorkerPo ol.java:103) - waiting to lock <0xdae67640> (a org.xmlBlaster.util.dispatch.DispatchWorkerPool) at org.xmlBlaster.util.dispatch.DispatchManager.startWorkerThread(DispatchM anager.java:722) - locked <0xdff773b8> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchManager.activateDispatchWorker(Disp atchManager.java:674) at org.xmlBlaster.util.dispatch.DispatchManager.setDispatchWorkerIsActive(D ispatchManager.java:888) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:150) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Thread.java:619) "XmlBlaster.DispatchWorkerPool.xmlBlaster_10_1_13_13_3412-5" daemon prio=10 tid=0x007a7000 nid=0x171 waiting for monitor entry [0xd3c5f000..0xd3c5f870] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.util.dispatch.DispatchWorkerPool.execute(DispatchWorkerPo ol.java:103) - waiting to lock <0xdae67640> (a org.xmlBlaster.util.dispatch.DispatchWorkerPool) at org.xmlBlaster.util.dispatch.DispatchManager.startWorkerThread(DispatchM anager.java:722) - locked <0xdd854e80> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchManager.activateDispatchWorker(Disp atchManager.java:674) at org.xmlBlaster.util.dispatch.DispatchManager.setDispatchWorkerIsActive(D ispatchManager.java:888) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:150) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Thread.java:619) "XmlBlaster.SubjectEntryShuffler" daemon prio=10 tid=0x007a9000 nid=0x170 in Object.wait() [0xd323f000..0xd323f9f0] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:485) at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.put(Unknown Source) - locked <0xe1a81d48> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$WaitWhenBlocked.blockedA ction(Unknown Source) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.execute(Unknown Source) at org.xmlBlaster.util.dispatch.DispatchWorkerPool.execute(DispatchWorkerPo ol.java:107) - locked <0xdae67640> (a org.xmlBlaster.util.dispatch.DispatchWorkerPool) at org.xmlBlaster.util.dispatch.DispatchManager.startWorkerThread(DispatchM anager.java:722) - locked <0xe16a3f48> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchManager.notifyAboutNewEntry(Dispatc hManager.java:652) at org.xmlBlaster.util.dispatch.DispatchManager.putPost(DispatchManager.jav a:570) at org.xmlBlaster.util.queue.ram.RamQueuePlugin.put(RamQueuePlugin.java:764 ) at org.xmlBlaster.authentication.SessionInfo.queueMessage(SessionInfo.java: 497) at org.xmlBlaster.authentication.SubjectInfo.forwardToSessionQueue(SubjectI nfo.java:659) at org.xmlBlaster.authentication.SubjectInfo.forwardToSessionQueue(SubjectI nfo.java:588) at org.xmlBlaster.engine.SubjectEntryShuffler.run(SubjectEntryShuffler.java :88) at java.lang.Thread.run(Thread.java:619) "XmlBlaster.DispatchWorkerPool.xmlBlaster_10_1_13_13_3412-4" daemon prio=10 tid=0x00c02c00 nid=0xcd waiting for monitor entry [0xd3b6f000..0xd3b6fbf0] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.util.dispatch.DispatchWorkerPool.execute(DispatchWorkerPo ol.java:103) - waiting to lock <0xdae67640> (a org.xmlBlaster.util.dispatch.DispatchWorkerPool) at org.xmlBlaster.util.dispatch.DispatchManager.startWorkerThread(DispatchM anager.java:722) - locked <0xdad7fe90> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchManager.activateDispatchWorker(Disp atchManager.java:674) at org.xmlBlaster.util.dispatch.DispatchManager.setDispatchWorkerIsActive(D ispatchManager.java:888) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:150) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Thread.java:619) "XmlBlaster.DispatchWorkerPool.xmlBlaster_10_1_13_13_3412-3" daemon prio=10 tid=0x00b76400 nid=0xcc waiting for monitor entry [0xd3b9e000..0xd3b9fa70] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.util.dispatch.DispatchWorkerPool.execute(DispatchWorkerPo ol.java:103) - waiting to lock <0xdae67640> (a org.xmlBlaster.util.dispatch.DispatchWorkerPool) at org.xmlBlaster.util.dispatch.DispatchManager.startWorkerThread(DispatchM anager.java:722) - locked <0xe21c2d50> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchManager.notifyAboutNewEntry(Dispatc hManager.java:652) at org.xmlBlaster.util.dispatch.DispatchManager.putPost(DispatchManager.jav a:570) at org.xmlBlaster.util.queue.ram.RamQueuePlugin.put(RamQueuePlugin.java:764 ) at org.xmlBlaster.engine.TopicHandler.invokeCallback(TopicHandler.java:1390 ) at org.xmlBlaster.engine.TopicHandler.invokeCallbackAndHandleFailure(TopicH andler.java:1222) at org.xmlBlaster.engine.TopicHandler.publish(TopicHandler.java:741) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1591) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1368) at org.xmlBlaster.engine.RequestBroker.publish(RequestBroker.java:1356) at org.xmlBlaster.engine.XmlBlasterImpl.publishArr(XmlBlasterImpl.java:198) at org.xmlBlaster.client.protocol.local.LocalConnection.publishArr(LocalCon nection.java:231) at org.xmlBlaster.client.dispatch.ClientDispatchConnection.publish(ClientDi spatchConnection.java:233) at org.xmlBlaster.client.dispatch.ClientDispatchConnection.doSend(ClientDis patchConnection.java:126) at org.xmlBlaster.util.dispatch.DispatchConnection.send(DispatchConnection. java:231) at org.xmlBlaster.util.dispatch.DispatchConnectionsHandler.send(DispatchCon nectionsHandler.java:455) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:70) at org.xmlBlaster.util.dispatch.DispatchManager.putPre(DispatchManager.java :561) at org.xmlBlaster.util.dispatch.DispatchManager.putPre(DispatchManager.java :535) at org.xmlBlaster.util.queue.ram.RamQueuePlugin.put(RamQueuePlugin.java:723 ) at org.xmlBlaster.client.XmlBlasterAccess.queueMessage(XmlBlasterAccess.jav a:831) at org.xmlBlaster.client.XmlBlasterAccess.publish(XmlBlasterAccess.java:991 ) at com.infovista.foundation.middleware.mom.xmlblaster.XBSession.publish(XBS ession.java:333) at com.infovista.foundation.middleware.mom.Session.send(Session.java:107) at com.infovista.foundation.jmx.event.LoginAcknowledger.loginMessage(LoginA cknowledger.java:223) at com.infovista.foundation.jmx.event.LoginAcknowledger.onMessage(LoginAckn owledger.java:123) at com.infovista.foundation.middleware.mom.xmlblaster.XBSession.update(XBSe ssion.java:257) at org.xmlBlaster.client.XmlBlasterAccess.update(XmlBlasterAccess.java:1082 ) at org.xmlBlaster.client.protocol.AbstractCallbackExtended.update(AbstractC allbackExtended.java:111) at org.xmlBlaster.client.protocol.AbstractCallbackExtended.update(AbstractC allbackExtended.java:199) at org.xmlBlaster.protocol.local.CallbackLocalDriver.sendUpdate(CallbackLoc alDriver.java:139) at org.xmlBlaster.engine.dispatch.CbDispatchConnection.doSend(CbDispatchCon nection.java:292) at org.xmlBlaster.util.dispatch.DispatchConnection.send(DispatchConnection. java:231) at org.xmlBlaster.util.dispatch.DispatchConnectionsHandler.send(DispatchCon nectionsHandler.java:455) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:114) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Thread.java:619) "XmlBlaster.DispatchWorkerPool.xmlBlaster_10_1_13_13_3412-2" daemon prio=10 tid=0x00c06000 nid=0xcb waiting for monitor entry [0xd3bcf000..0xd3bcfb70] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.util.dispatch.DispatchWorkerPool.execute(DispatchWorkerPo ol.java:103) - waiting to lock <0xdae67640> (a org.xmlBlaster.util.dispatch.DispatchWorkerPool) at org.xmlBlaster.util.dispatch.DispatchManager.startWorkerThread(DispatchM anager.java:722) - locked <0xdf576920> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchManager.activateDispatchWorker(Disp atchManager.java:674) at org.xmlBlaster.util.dispatch.DispatchManager.setDispatchWorkerIsActive(D ispatchManager.java:888) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:150) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Thread.java:619) "XmlBlaster.DispatchWorkerPool.xmlBlaster_10_1_13_13_3412-1" daemon prio=10 tid=0x00c05800 nid=0xca waiting for monitor entry [0xd3bff000..0xd3bff8f0] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.util.dispatch.DispatchWorkerPool.execute(DispatchWorkerPo ol.java:103) - waiting to lock <0xdae67640> (a org.xmlBlaster.util.dispatch.DispatchWorkerPool) at org.xmlBlaster.util.dispatch.DispatchManager.startWorkerThread(DispatchM anager.java:722) - locked <0xe052bdd8> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchManager.activateDispatchWorker(Disp atchManager.java:674) at org.xmlBlaster.util.dispatch.DispatchManager.setDispatchWorkerIsActive(D ispatchManager.java:888) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:150) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Thread.java:619) "XmlBlaster.DispatchWorkerPool.xmlBlaster_10_1_13_13_3412-0" daemon prio=10 tid=0x00c05400 nid=0xc9 waiting for monitor entry [0xd3c2f000..0xd3c2f870] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.util.dispatch.DispatchWorkerPool.execute(DispatchWorkerPo ol.java:103) - waiting to lock <0xdae67640> (a org.xmlBlaster.util.dispatch.DispatchWorkerPool) at org.xmlBlaster.util.dispatch.DispatchManager.startWorkerThread(DispatchM anager.java:722) - locked <0xde036a68> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchManager.activateDispatchWorker(Disp atchManager.java:674) at org.xmlBlaster.util.dispatch.DispatchManager.setDispatchWorkerIsActive(D ispatchManager.java:888) at org.xmlBlaster.util.dispatch.DispatchWorker.run(DispatchWorker.java:150) at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Thread.java:619) "XmlBlaster.MessageTimer" daemon prio=10 tid=0x0137cc00 nid=0xc6 waiting for monitor entry [0xd3cbf000..0xd3cbfaf0] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.engine.MsgUnitWrapper.toExpired(MsgUnitWrapper.java:521) - locked <0xd83daa60> (a org.xmlBlaster.engine.MsgUnitWrapper) at org.xmlBlaster.engine.MsgUnitWrapper.timeout(MsgUnitWrapper.java:594) at org.xmlBlaster.util.Timeout.run(Timeout.java:189) "XmlBlaster.TopicTimer" daemon prio=10 tid=0x00c01400 nid=0xc2 in Object.wait() [0xd40ff000..0xd40ff8f0] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.xmlBlaster.util.Timeout.run(Timeout.java:203) - locked <0xdacbf2f8> (a org.xmlBlaster.util.Timeout) "XmlBlaster.PingTimer" daemon prio=10 tid=0x0137d800 nid=0xc1 waiting for monitor entry [0xd48af000..0xd48af870] java.lang.Thread.State: BLOCKED (on object monitor) at org.xmlBlaster.util.dispatch.DispatchConnection.handleTransition(Dispatc hConnection.java:523) - waiting to lock <0xdff773b8> (a org.xmlBlaster.util.dispatch.DispatchManager) at org.xmlBlaster.util.dispatch.DispatchConnection.ping(DispatchConnection. java:350) at org.xmlBlaster.util.dispatch.DispatchConnection.timeout(DispatchConnecti on.java:423) at org.xmlBlaster.util.Timeout.run(Timeout.java:189) "XmlBlaster.PingTimer" daemon prio=10 tid=0x008d7400 nid=0xb4 in Object.wait() [0xd3d6f000..0xd3d6fbf0] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.xmlBlaster.util.Timeout.run(Timeout.java:203) - locked <0xdacbe8a8> (a org.xmlBlaster.util.Timeout) "XmlBlaster.HttpIORServer" daemon prio=10 tid=0x0130c000 nid=0x66 runnable [0xd4baf000..0xd4bafaf0] java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) - locked <0xda94f2d8> (a java.net.SocksSocketImpl) at java.net.ServerSocket.implAccept(ServerSocket.java:453) at java.net.ServerSocket.accept(ServerSocket.java:421) at org.xmlBlaster.util.http.HttpIORServer.run(HttpIORServer.java:157) "XmlBlaster.SocketDriver" daemon prio=10 tid=0x01309c00 nid=0x65 runnable [0xd4bdf000..0xd4bdfa70] java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) - locked <0xda94f1e8> (a java.net.SocksSocketImpl) at java.net.ServerSocket.implAccept(ServerSocket.java:453) at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketI mpl.java:259) at org.xmlBlaster.protocol.socket.SocketDriver.run(SocketDriver.java:533) "XmlBlaster.SocketDriver" daemon prio=10 tid=0x013a7000 nid=0x64 runnable [0xd4c0f000..0xd4c0fbf0] java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384) - locked <0xda94f260> (a java.net.SocksSocketImpl) at java.net.ServerSocket.implAccept(ServerSocket.java:453) at java.net.ServerSocket.accept(ServerSocket.java:421) at org.xmlBlaster.protocol.socket.SocketDriver.run(SocketDriver.java:533) "XmlBlaster.SessionTimer" daemon prio=10 tid=0x013a7800 nid=0x63 in Object.wait() [0xd4c3f000..0xd4c3fb70] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.xmlBlaster.util.Timeout.run(Timeout.java:203) - locked <0xda949870> (a org.xmlBlaster.util.Timeout) "XmlBlaster.Consumer" daemon prio=10 tid=0x003fe400 nid=0x62 in Object.wait() [0xd4c6f000..0xd4c6f8f0] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0xdac36698> (a edu.emory.mathcs.backport.java.util.concurrent.LinkedBlockingQueue$Seria lizableLock) at java.lang.Object.wait(Object.java:485) at edu.emory.mathcs.backport.java.util.concurrent.LinkedBlockingQueue.take( LinkedBlockingQueue.java:316) - locked <0xdac36698> (a edu.emory.mathcs.backport.java.util.concurrent.LinkedBlockingQueue$Seria lizableLock) at org.xmlBlaster.engine.TopicAccessor$Consumer.run(TopicAccessor.java:451) at java.lang.Thread.run(Thread.java:619)

--
Marcel Ruff
http://www.xmlBlaster.org

References:
- Re: [xmlblaster] dead lock
  - From: "David Robison" <drrobison at openroadsconsulting.com>
- RE: [xmlblaster] dead lock
  - From: "Xavier Roques" <xRoques at infovista.com>
- Re: [xmlblaster] dead lock
  - From: Marcel Ruff <mr at marcelruff.info>
- [xmlblaster] server hangs when changing the size maximum of the DispathcWorkerPool
  - From: "Xavier Roques" <xRoques at infovista.com>

Prev by Date: [xmlblaster] server hangs when changing the size maximum of the DispathcWorkerPool
Next by Date: RE: [xmlblaster] dead lock
Previous by thread: [xmlblaster] server hangs when changing the size maximum of the DispathcWorkerPool
Next by thread: Re: [xmlblaster] dead lock
Index(es):
- Date
- Thread