Martin recently
announced version 2.0 of
the Disruptor - basically there have been so many changes since we first open-sourced it that it's time to mark that officially. His post goes over all the changes, the aim of this article is to attempt to translate my previous blog posts into new-world-speak, since it's going to take a long time to re-write each of them all over again. Now I see the disadvantage of hand-drawing everything.
In the old world
This is an example of a configuration of the Disruptor (specifically a diamond configuration). If none of this means anything to you, feel free to go back and refresh yourself on all the (now outdated)
Disruptor details.
The most obvious changes over the last few weeks have been:
- Updated naming convention
- Integrating the producer barrier into the ring buffer
- Adding the Disruptor wizard into the main code base.
The New World Order
You'll see the fundamentals are pretty much the same. It's simpler, because the ProducerBarrier
is no longer an entity in its own right - its replacement is the PublishPort
interface, which is implemented by the RingBuffer
itself.
Similarly the name DependencyBarrier
instead of ConsumerBarrier
clarifies the job of this object; Publisher
(instead of Producer
) and EventProcessor
instead of Consumer
also more accurately represent what these things do. There was always a little confusion over the name Consumer
, since consumers never actually consumed anything from the ring buffer. It was simply a term that we hoped would make sense to those who were used to queue implementations.
Not shown on the diagram is the name change of the items in the RingBuffer
- in the old world, we called this Entry
, now they're an Event
, hence EventProcessor
at the other end.
The aim of this wholesale rename has not been to completely discredit all my old blogs so I can continue blogging about the Disruptor
ad infinitum. This is far from what I want - I have other, more fluffy, things to write about. The aim of the rename is to make it easier to understand how the Disruptor works and how to use it. Although
we use the Disruptor for event processing, when we open sourced it we wanted it to look like a general purpose solution, so the naming convention tried to represent that. But in fact the event processing model does seem more intuitive.
No more tedious wiring
Now the
Disruptor wizard is part of the Disruptor itself, my whole
post on wiring is pretty pointless - which is good, actually, because it was a little involved.
These days, if you want to create the diamond pattern (for example the
FizzBuzz performance test), it's a lot simpler:
DisruptorWizard dw = new DisruptorWizard<FizzBuzzEvent>(
ENTRY_FACTORY,
RING_BUFFER_SIZE,
EXECUTOR,
ClaimStrategy.Option.SINGLE_THREADED,
WaitStrategy.Option.YIELDING);
FizzBuzzEventHandler fizzHandler =
new FizzBuzzEventHandler(FIZZ);
FizzBuzzEventHandler buzzHandler =
new FizzBuzzEventHandler(BUZZ);
FizzBuzzEventHandler fizzBuzzHandler =
new FizzBuzzEventHandler(FIZZ_BUZZ);
dw.handleEventsWith(fizzHandler, buzzHandler)
.then(fizzBuzzHandler);
RingBuffer ringBuffer = dw.start();
Note there is a
Wiki page on the Disruptor Wizard.
Other changes: performance improvements
As Martin mentions in
his post, he's managed to significantly improve the performance (even more!) of the Disruptor in 2.0.
The short version of this is that there is a shiny new class,
Sequence
, which both takes care of the
cache line padding, and removes the need for
memory barriers. The cache line padding is now done slightly differently because, bless Java 7's little cotton socks, it managed to "optimise" our old technique away.
I'll leave you to read the details over there, in this post I just wanted to give a quick summary of the changes and explain why my old diagrams may no longer be correct.
Trisha, your blogs have been great. The new naming scheme makes it MUCH easier to understand what it going on.
ReplyDeleteThank you!
ReplyDeleteHi trisha,
ReplyDeleteI try do understand the advantage to use the disruptor for me algo trader for reduce my latency ?!
There advantage or it's just for the server side (LMAX) ?
Thanks.
Regards.
It's not just a server-side thing. You can use it anywhere you would usually use a queue, but you can also use it anywhere you want to run processes in parallel. So there are definitely use-cases for a trading application.
ReplyDeleteWe don't have any examples of this right now, but I know there are people working on using the Disruptor for a trading app.
When processing events with a chain in the 'diamond' pattern above, is there something special that the intermediate handlers have to do to insure that downstream handlers get their chance to process the events?
ReplyDeleteI have put together a simple example, but only my first handler is getting invoked, none of the rest of the handlers.
@kov you need to remember to increment the sequence number when you've finished processing. The downstream consumers will be reading that to see if they can process anything else.
ReplyDeleteThanks, Trisha. Looking at trunk, AbstractEvent.setSequence is package-scoped; is there something in the framework that sets sequence for me? Otherwise I would have to declare all my event-types within your package.
ReplyDeleteOh ... hang on, I'm not making any sense; I assume I should not change the sequence on an event.
ReplyDeleteWhat sequence should I be increasing? From what class is it accessed?
Each consumer tracks its own sequence. It knows which number it needs to access from the ring buffer, so you need to keep a field in your consumer to track it.
ReplyDeleteIf you need technical help, the best place to go is the google group: https://groups.google.com/forum/#!forum/lmax-disruptor
Hi Trisha, great blog and great posts about the Disruptor. Shame that you are not at LMAX anymore, I will find a lot harder to understand what's in the next big release of the Disruptor without your drawings to help.. :(
ReplyDeleteSpeaking of the Disruptor I was wondering if I could pick your brain regarding a use case I had in mind. I am currently coding an order book implementation (for a job application) and was thinking that it would be great if I used the Disruptor somewhere in my code. However I am unsure where it would be best to use it and was wondering if you could give some guidance.
Basically there are many order books but only one source of events (the orders publisher). It seems logical that my Disruptor should hold the orders events right? My question then is should I use one Disruptor per order book (the event consumer) or have all the order books read off the same Disruptor (could be several hundreds order books listening)? Or should I put several Disruptors in series maybe? Those maybe a bit vague questions but any input would be hugely helpful! Thanks Trisha and happy new year! Julien
A nice little pattern this disruptor, thanks for explaining it so well Trisha. Quick question: do you think it's possible to turn a disruptor into an order matching algo?
ReplyDeleteI don't think so, it's designed for passing messages around not doing the processing. You can build a matcher and have it as a processor, but that's not at all dependent upon being part of the Disruptor.
Delete