Skip to content

Delivery, settlement and flow control

Source: source.go. This page explains what the adapter does with a message between the broker and your handler. The core's contract for it is in the backend reference and Settle a delivery.

Credit is the bound

Each subscription is one receiver link on a session of its own. The adapter takes control of the link's credit rather than leaving it to the library: it issues Bound credit when the link attaches, and every transfer received uses one.

A credit is given back exactly when the core frees the slot:

  • under AtLeastOnce, at settlement: accepted, rejected, modified, or handed on by the lease;
  • under AtMostOnce, at hand-over to the core, because the message was accepted before it got that far.

So the messages buffered in the adapter plus the credit outstanding never exceed Bound. The adapter's buffer holds Bound - 1, and the delivery being handed over is the last one. A slow handler therefore stops the broker sending, and what it does not send stays in the queue. Nothing is shed. Refused and Evicted are zero by construction, which is why Overflow is empty in Capabilities.

Pending, as SourceStats.Pending, is the buffer plus what is being handed over.

If the broker will not take the credit (IssueCredit fails), the adapter logs messaging-amqp: issuing credit failed with the subscription's name and carries on.

Settling

The adapter maps the core's outcomes to AMQP's.

The core says AMQP outcome Effect on the broker
Ack accepted the message is done.
a decline, with attempts left modified, delivery-failed true the broker counts a failed delivery and redelivers.
a decline on the last attempt (Attempt >= MaxDeliver) rejected the broker moves the message to the address's dead-letter address.
the lease reaching its bound the same two, by attempt as above.

A decline is a Nack, or silence: the core settles a silent handler as a decline under AtLeastOnce. When it settles on the last attempt the adapter counts the source's Exhausted.

The tutorial shows the last row: with MaxDeliver 2, the handler sees attempts 1 and 2 and the message is on DLQ after the second decline.

Attempt and redelivery

Attempt is the message's AMQP delivery-count plus one, or 1 if the message has no header. The count is the broker's, so it survives a restart on a broker configured to persist it, and MaxDeliver bounds a message, not one process's memory of it.

Redelivery is the broker's. The adapter asks for no delay (DelaysRedelivery is false), so a declined message is available again at once, subject to whatever redelivery delay the address is configured with on the broker; the adapter neither reads nor declares that.

MaxDeliver is the adapter's, with the broker as a backstop

The adapter rejects on the last attempt itself. It also checks at subscribe that the address's max-delivery-attempts is not below MaxDeliver, so the broker never dead-letters first and the Exhausted count stays true. See Topology it expects.

Late and refused settlements

  • After the lease has handed a message on, the handler's later Ack or Nack is a no-op that returns nil. It does not return the credit a second time.
  • After the source is retired, a settlement is refused with messaging.ErrSettlementRefused.
  • A failed broker call comes back wrapped: messaging-amqp: accepting the message, rejecting the message or handing back the message.

The lease

The lease is the consumer's own timer. AMQP 1.0 gives an unsettled delivery no lease of its own: a broker holds the message while the link is up. So the adapter's ExtendsLease is true because holding is extending. There is no heartbeat to send.

SlowHandler Timer length At the bound
Redeliver the subscription's AckWait, or Settings.DefaultAckWait (30 s) if zero the message is handed on; nothing is counted
ExtendLease MaxProcessingTime the message is handed on and LeaseExpired counts it

"Handed on" is the decline row above: modified with delivery-failed, or rejected if it was the last attempt. A bound of zero or less sets no timer. AtMostOnce has no timer.

The timer starts when the adapter reads the message off the link, not when a handler starts, so time spent in the adapter's buffer counts against it. A handler slow enough to keep messages queued behind it can see their leases expire before they are delivered. See Limits and gaps.

Retiring a subscription

Retirement stops reading, settles everything unsettled, closes Deliveries, and ends the source's own session. Ending the session lets the link go without unsubscribing: closing the receiver would delete a durable subscription on Artemis.

Where the message is What happens
in the adapter's buffer, never handed over modified without delivery-failed: returned, uncounted. A plain release is avoided because Artemis decrements its persisted count on release.
with a handler modified with delivery-failed: returned, counted as an attempt.
AtMostOnce nothing: it was accepted before hand-off and is simply not delivered here.

If handing a message back fails, the adapter logs it and the link close leaves the message to the broker. Credit the link still held when the session ends is the broker's to requeue, uncounted.

If the link goes away on its own (a detach, a closed session, a broker that removed the queue), the adapter logs messaging-amqp: the receiver is gone and ends the source with the cause, which the core reads from the source's final.

At-most-once

Each message is accepted, and the receipt waited for, before it is buffered or handed over. A competing member that takes a message and retires without a word leaves nothing for the next member. If the accept fails the message is not handed over, a warning is logged, and its credit is returned.

A message no format can read

A received message goes through envelope.Read with the Reads chain before anything else. One that fails is unreadable, in any of three ways: no format claims it, two formats both claim it (the reader refuses to guess between them), or one claims it and cannot decode it. It never becomes a Delivery and never reaches a handler.

What happens instead, every time it is received:

  1. The adapter logs at warn level, messaging-amqp: a message on the queue could not be read, with the subscription's name, the address and the reader's error.
  2. It adds one to SourceStats.Unreadable, which the bus folds into Stats(name).Unreadable. Unreadable counts attempts.
  3. It settles the message the way the mode settles every message:
  4. under AtLeastOnce, as a decline: modified with delivery-failed while it has attempts left, and rejected to the dead-letter address on its last;
  5. under AtMostOnce, accepted, and so dropped. Without the accept it would come back.
  6. It returns the message's credit, whether or not the settlement succeeded, so unreadable messages cannot shrink the window. Once retirement has begun it returns none, because a credit then would only draw another transfer to give back.

It is not counted in Accepted, which counts where a message enters the queue after the read, nor in Exhausted, even when it is rejected on its last attempt. It was never offered to the bus.

Under AtLeastOnce one unreadable message therefore adds MaxDeliver to Unreadable before it dead-letters, and is then on the dead-letter queue, where any AMQP client can read it. See Find messages no format could read.

If the settlement itself fails, the adapter logs messaging-amqp: settling a message that could not be read failed and returns the credit all the same, as it does for a failed settlement of any message. A failure can mean the link is going, and then the receive loop ends the source when it finds the link gone. It can also mean only that the call outlasted its 10s budget on a live but busy link, where withholding the credit would stall the source once Bound such failures had built up.

The settlement runs on a context that retirement does not cancel, because it is also the give-back during retirement.

The conformance row that holds all of this is AnUnreadableMessageNeverReachesAHandler. The fixture sends one message of each kind, under all four formats, to an address with its own client: see Run the conformance suite. The design is spec 0015.