Rebouncer is a Go package offering a powerful and flexible engine for taking in a noisy source of events and producing a cleaner source. It takes its name from the concept of a Debouncer, which is an electrical engineering term used to describe the process of taking a bunch of events (samples), removing the extraneous or undesired ones, and emitting just one. For example, debouncing is how we get to use our keyboards to create one keypress event at a time, even though the underlying circuits are registering many events in quick succession.

Rebouncer takes this concept further. It handles the concurrent mechanics of a “dirty” channel and a “clean” channel, allowing you to decide what to pop on the clean channel. One use-case might be to simply batch up events once per second, and emit them periodically in packets. One might be to filter messages and only return relevant ones, reducing noise. In a particular use-case, a clean channel might have more traffic than a dirty one, or the clean channel could feed back events to the dirty channel, creating a generative process that evolves over time.

Rebouncer accomplishes this with well-known lifecycles (Ingest, Map, Reduce, Quantize, Emit, Egest), and an ability to inject user-defined functions containing business-logic, nicely encapsulated through the magic of closures. For common cases, the library comes with default convenience lifecycle functions. For advanced cases, you write your own and instantiate like so:

config := rebouncer.Config{
    ingestor: myIngestorFunc,
    Reducer: myReducerFunc,
    Quantizer: myQuantizerFunc,
}

myRebouncer := rebouncer.New(config)

//  here's a channel you can listen on
myCleanChannel := myRebouncer.Subscribe()

A common case for an engine like this is to power a live-reload server, such as Fasthak. Fasthak uses convience functions to invoke Rebouncer:

stateMachine := rebouncer.NewInotify(*watchDir, 1000)
niceEvents := stateMachine.Subscribe()

That was easy! But we want to know more about what’s happening under the hood, and how we might use Rebouncer for less common cases. Let’s take a look at the basic architectural components.

Primitives

Rebouncer expects to always work with events in its own format (except at ingestion, but we’ll see later how we keep information safely hidden), allowing for composition of arbitrary complexity. The most atomic unit is a NiceEvent

//  An event with a nice expected structure
type NiceEvent struct {
    id uint64
    Topic string
    Data interface{} // data necessary for your use-case
}

plus some extras that are there to make reasoning about the system easier

// Queue is a slice of NiceEvents
type Queue []NiceEvent

// NiceChannel is a channel of NiceEvents
type NiceChannel chan NiceEvent

type NiceEvent[T any] struct {
    id uint64
    Topic string
    Data T
}

Ingestor

Ingest is the first lifecycle event. Ingestor is the only component that has access to “dirty” events. It cleans them and exposes them to the rest of the app through a NiceChannel. This is an internal channel only available to Ingestor and only used to asynchronously push events to the Queue, even as the Queue is being drained. It is essentially a message bus.

// consumes dirty events, cleans them, and sends to a NiceChannel
type ingestor func() chan NiceEvent

Any time Ingestor pushes an event to the Queue, Reducer is run

Reducer

The second lifecycle event. It operates on the entire queue, and modifies it. Commonly (but not necessarily) by removing extraneous events or combining two to one.

type Reducer func([]NiceEvent) []NiceEvent
// or if you prefer
type Reducer func(Queue) Queue

Now with a nicely curated batch of events, our business logic needs to decide whether it’s time to Emit().

Quantizer

Enter Quantizer. The third lifecycle event. Looks at the Queue and maybe some other state it doesn’t share with Rebouncer. In the live-reloader case, this is a simple timer running once per second (or some other t), deciding it’s time to Emit() if there are any events in the queue at all.

type Quantizer func(chan bool, *[]NiceEvent)

The channel it will listen on is exposed by Rebouncer as readyChannel. If readyChannel gets a true, it’s time to Emit().

Emit

Fourth and final lifecycle event before the whole thing starts again. There is no user function for this. Emit() simply flushes all the events in the Queue to the consumer.

Information Flow

The user-defined functions get to keep their own state, and they do not get access to the internals, except for what’s explicitly passed in via arguments. The following illustration shows how information flows through the system. The user-defined functions are represented as teal rectangles with braces inside, with tendrils emanating into a well-contained private space, indicating that they are closures with access to their own state. This means your business logic gets to be nicely isolated, but as powerful and broad as need be.

At the heart of Rebouncer’s architecture is the Queue, which is protected by a mutex, here shown as a padlock. Operations that want to write to the queue pile up on their respective channels (represented with Ⓒ) until they can do so. The channels are well-buffered to permit asynchronous work to occur.

The first lifecycle event is carried out by the Ingestor, portrayed here as a beast eating flies, symbolising it’s voracious hunger for dirty events. It consumes them speedily, digests them, and evacuates them to the incomingEvents channel. These events are pushed onto the Queue as soon as they can be (they may have to wait for a mutex unlock). Ingestor has no access to any of Rebouncer’s data other than the channel it writes to.

Next is Reducer, here represented as a triangle. It is able to see events on the queue. It’s final act, once it’s carried out all necessary business logic, is to write back to the queue a normalized set of events. Many of the events it operates on it may have already seen. That’s okay. It still gets to see all of them and decide what to do. Reducer takes a lock, guaranteeing no events are lost.

Quantizer runs after Reducer has written to the Queue. It contains its own state and decides when to tell Rebouncer to Emit(). One of the components of it’s private memory space could well be a clock, as in the case of typical Debouncer powering a live-reloader. Quantizer can see the Queue but cannot affect it. It can however send signal to a private channel (called readyChan). This is the mechanism that allows Rebouncer to know it’s time to Emit(). Emit() sends all the messages in the Queue to the main exit-point for consuming apps: A channel of NiceEvents.

The consuming process (from Rebouncer’s point of view, this is the Egest lifecycle event) continually and happily consumes nicely curated, filtered, and quantized events. This is represented by a happy blue pacman type fellow with a more discerning taste and less ravenous hunger than the diabolical Ingestor.

// Egest
for niceEvent := range myRebouncerInstance.Subscribe() {
    // do something
}