Skip to content

Consume from RabbitMQ

By the end of this you will have a RabbitMQ broker with the exchange and queues the adapter expects, a bus consuming from it at-least-once, and a message that exhausted its attempts sitting in the dead-letter queue.

You need Docker and Go. The adapter creates no durable topology, so the first step is to make it yourself, the way an operator would. The Go program was written against the adapter's source and the topology steps were run against RabbitMQ 4.3.5; the program itself was not run for this page.

1. Start a broker

docker run -d --name rmq -p 5672:5672 -p 15672:15672 rabbitmq:4.3.5-management

The AMQP URL in the program below carries no credentials, so amqp.Dial uses the image's guest account. Wait until http://localhost:15672 answers. The management API must be reachable: Open calls /api/overview before it dials AMQP and fails with ErrManagement if it does not answer.

2. Make the topology

The adapter needs a topic exchange, a quorum queue bound to it, and a dead-letter exchange and queue for the quorum queue. Each is created here through the management API, with the default guest account of the image.

M=http://localhost:15672/api
H='Content-Type: application/json'
AUTH=guest:guest # the image's default account

# the subject space: a topic exchange
curl -u "$AUTH" -H "$H" -X PUT $M/exchanges/%2F/events -d '{"type":"topic","durable":true}'

# somewhere for exhausted messages to go
curl -u "$AUTH" -H "$H" -X PUT $M/exchanges/%2F/orders.dlx -d '{"type":"fanout","durable":true}'
curl -u "$AUTH" -H "$H" -X PUT $M/queues/%2F/orders.dlq -d '{"durable":true,"arguments":{"x-queue-type":"quorum"}}'
curl -u "$AUTH" -H "$H" -X POST $M/bindings/%2F/e/orders.dlx/q/orders.dlq -d '{"routing_key":""}'

# the queue the bus will consume
curl -u "$AUTH" -H "$H" -X PUT $M/queues/%2F/orders-worker -d '{
  "durable": true,
  "arguments": {
    "x-queue-type": "quorum",
    "x-dead-letter-exchange": "orders.dlx",
    "x-dead-letter-strategy": "at-least-once",
    "x-overflow": "reject-publish",
    "x-delivery-limit": 4
  }
}'
curl -u "$AUTH" -H "$H" -X POST $M/bindings/%2F/e/events/q/orders-worker -d '{"routing_key":"orders.#"}'

Three things in orders-worker are not optional: it is a quorum queue, it dead-letters with at-least-once strategy and reject-publish overflow, and its delivery limit is at least MaxDeliver - 1. The next step asks for MaxDeliver: 5, so the limit is 4. Its binding, orders.#, is the broker's spelling of the subscription pattern orders.>. Topology it expects lists every check.

3. Write the program

package main

import (
    "context"
    "log"
    "os/signal"
    "syscall"
    "time"

    "gitlab.com/phpboyscout/go/envelope"
    "gitlab.com/phpboyscout/go/messaging"
    rabbitmq "gitlab.com/phpboyscout/go/messaging-rabbitmq"
)

func main() {
    ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
    defer stop()

    backend := rabbitmq.New(rabbitmq.Settings{
        URL:        "amqp://localhost:5672/",
        Management: rabbitmq.Management{URL: "http://localhost:15672", Username: "guest", Password: "guest"},
        Exchange:   "events",
    })

    bus, err := messaging.New(backend, messaging.Settings{
        Subscriptions: []messaging.SubscriptionSpec{{
            Name:        "orders",
            Pattern:     "orders.>",
            Shape:       messaging.Fanout,
            Mode:        messaging.AtLeastOnce,
            DurableName: "orders-worker",
            MaxDeliver:  5,
            Handler: func(ctx context.Context, d messaging.Delivery) error {
                log.Printf("attempt %d of %s: %s", d.Attempt, d.Subject, d.Event.Data)

                if string(d.Event.Data) == `{"poison":true}` {
                    return d.Nack()
                }

                return d.Ack()
            },
        }},
    })
    if err != nil {
        log.Fatal(err)
    }

    if err := bus.Start(ctx); err != nil {
        log.Fatal(err)
    }
    defer bus.Stop(context.Background())

    select {
    case <-bus.Ready():
    case <-time.After(30 * time.Second):
        log.Fatal("the bus did not become ready")
    }

    for _, data := range []string{`{"id":1}`, `{"poison":true}`} {
        err := bus.Publish(ctx, "orders.created", envelope.Event{
            SpecVersion: "1.0",
            ID:          "order-" + data,
            Source:      "urn:example:tutorial",
            Type:        "com.example.order.created.v1",
            Data:        []byte(data),
        })
        if err != nil {
            log.Fatal(err)
        }
    }

    <-ctx.Done()
}

DurableName is the queue's name: the default Topology, ByName, returns it unchanged. The subscription declares nothing on the broker. bus.Start returns once Subscribe has found orders-worker and verified it; if the queue is wrong it returns the refusal instead.

4. Run it

go run .

The first event is received once and acknowledged. The second is declined on every attempt. Each decline of an attempt below MaxDeliver is a basic.reject with requeue, so the broker puts the message back and counts the failed delivery; the handler sees attempt 1 through attempt 5. The fifth decline is a basic.reject without requeue, which dead-letters the message.

5. Look at the dead-letter queue

In the shell from step 2, where AUTH is set:

curl -s -u "$AUTH" http://localhost:15672/api/queues/%2F/orders.dlq | grep -o '"messages":[0-9]*'

The management API refreshes queue statistics every few seconds, so the count may take a moment to show the one message. It is in orders.dlq, and the bus has counted it Exhausted in bus.Stats("orders").

What you have

A bus on a verified quorum queue, with the settlement contract of Settle a delivery running on RabbitMQ's own rejection and dead-lettering.

Clean up with docker rm -f rmq. Next: how a message is settled says exactly what each outcome does on the broker, and prepare the exchange and queues is the checklist for a real one.