[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [xmlblaster] Newbie: xmlblaster and large messages/recovery of partial transfers
Darrell Berry wrote:
>Just been pointed at xmblaster. I'm working on a distributed delivery
>mechanism for large files (>100Mb, probably most around 500Mb, some up
>to 2 or 3 Gb)
>Xmlblaster seems to offer a lot of what I need -- straightforward
>wrapping of content and metadata, infrastructure for both broadcast and
>p2p communications). However two questions:
>- Is xmlblaster robust and efficient for messages of this size?
No, you should drive some tests yourself to find out.
Here is a command line based test, start the three processes in three
(set CLASSPATH first)
java org.xmlBlaster.client.reader.SubscribeMessage -oid bigtest
java org.xmlBlaster.client.feeder.PublishFile -c MyBigFile.txt
-xmlKey "<key oid='bigtest'/>" -xmlQos "<qos/>"
Provide one of your big files and rename it to MyBigFile.txt before.
>- I see that you have a mechanism for making messages persist in the
>face of the system crashing or presumably one end of a connection going
>going> SO far so good -- however for messages of this size I would want
>to be able to resume transfer from the point at which contact broke off
>--if I've tranferred 90% of a 1Gb message and the receiver gets pulled
>from the network, when it resumes I would want to only have to download
>the remaining 10% -- have you any thoughts on this?
>Thx -- looks interesting!
I believe having a 3 GB file in RAM will be a problem. The messages is
sent in one piece.
You could add a little java client which divides the file into 10 MB
pieces, and the receiver
concatenates them again.
(Hmm, this would be a nice new feature in the xmlBlaster client lib -
eventually in future -)
keep us informed about your tests ...