Skip to content

Publish and receive through SQS

By the end of this you will have an SNS topic, an SQS queue behind it with a dead-letter queue and a redrive policy, and a Go program that publishes an event to the topic and receives it from the queue through the adapter.

It needs Docker and the AWS CLI. Nothing here touches a real AWS account: LocalStack stands in for it, at the version the adapter's own suite runs against.

1. Start LocalStack

$ docker run -d --rm --name sqs-tutorial -p 127.0.0.1:4566:4566 localstack/localstack:4.14.0

Point the CLI, and later the Go program, at it. LocalStack accepts any credentials:

$ export AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test AWS_REGION=us-east-1
$ export AWS_ENDPOINT_URL=http://127.0.0.1:4566

2. Create the topology

The adapter creates none of this for a durable subscription, so you do. The subject will be orders.created. The adapter derives the topic name by replacing each dot with an underscore, so the topic is orders_created. The queue is named by the subscription's DurableName, here billing. See Topology and naming.

$ TOPIC=$(aws sns create-topic --name orders_created --query TopicArn --output text)

$ DLQ=$(aws sqs create-queue --queue-name billing-dlq --query QueueUrl --output text)
$ DLQARN=$(aws sqs get-queue-attributes --queue-url $DLQ \
    --attribute-names QueueArn --query Attributes.QueueArn --output text)

$ Q=$(aws sqs create-queue --queue-name billing --attributes \
    "{\"VisibilityTimeout\":\"30\",\"RedrivePolicy\":\"{\\\"deadLetterTargetArn\\\":\\\"$DLQARN\\\",\\\"maxReceiveCount\\\":\\\"5\\\"}\"}" \
    --query QueueUrl --output text)
$ QARN=$(aws sqs get-queue-attributes --queue-url $Q \
    --attribute-names QueueArn --query Attributes.QueueArn --output text)

$ aws sns subscribe --topic-arn $TOPIC --protocol sqs --notification-endpoint $QARN \
    --attributes RawMessageDelivery=true

Three things there matter to the adapter. The queue has a redrive policy whose maxReceiveCount is 5, which the subscription's MaxDeliver must equal. The queue is subscribed to the topic, which the adapter does not check. And the subscription is raw: the adapter reads the SQS message's body and attributes as SNS published them, so a subscription that wraps each message in SNS's JSON envelope would make every message unreadable.

3. Write the program

package main

import (
    "context"
    "log"
    "log/slog"
    "os"
    "time"

    "gitlab.com/phpboyscout/go/awsclient"
    "gitlab.com/phpboyscout/go/envelope"
    "gitlab.com/phpboyscout/go/messaging"

    sqs "gitlab.com/phpboyscout/go/messaging-sqs"
)

func main() {
    ctx := context.Background()

    backend := sqs.Standard(sqs.Settings{
        AWS:    awsclient.Ambient(awsclient.WithRegion("us-east-1")),
        Logger: slog.New(slog.NewTextHandler(os.Stderr, nil)),
    })

    got := make(chan messaging.Delivery, 1)

    bus, err := messaging.New(backend, messaging.Settings{
        Source: "urn:example:tutorial",
        Subscriptions: []messaging.SubscriptionSpec{{
            Name:        "billing",
            Pattern:     "orders.created",
            Shape:       messaging.Fanout,
            Mode:        messaging.AtLeastOnce,
            MaxDeliver:  5,
            DurableName: "billing",
            Handler: func(ctx context.Context, d messaging.Delivery) error {
                got <- d

                return d.Ack()
            },
        }},
    })
    if err != nil {
        log.Fatal(err)
    }

    if err := bus.Start(ctx); err != nil {
        log.Fatal(err)
    }

    defer bus.Stop(context.Background())

    select {
    case <-bus.Ready():
    case <-time.After(30 * time.Second):
        log.Fatal("the bus did not become ready")
    }

    err = bus.Publish(ctx, "orders.created", envelope.Event{
        ID:              "order-4172",
        Source:          "urn:example:tutorial",
        Type:            "com.example.order.created.v1",
        DataContentType: "application/json",
        Data:            []byte(`{"order":"4172"}`),
    })
    if err != nil {
        log.Fatal(err)
    }

    select {
    case d := <-got:
        log.Printf("received %s on %s, attempt %d: %s", d.Event.ID, d.Subject, d.Attempt, d.Event.Data)
    case <-time.After(30 * time.Second):
        log.Fatal("nothing arrived")
    }
}

The pattern is orders.created, an exact subject. A pattern with * or > is refused, because SQS has no pattern language. The subscription asks for AtLeastOnce, which both varieties offer, and names a DurableName, which the adapter uses as the queue name. The Region is passed because the adapter never guesses one.

4. Run it

$ go run .
2026/10/31 03:50:58 received order-4172 on orders.created, attempt 1: {"order":"4172"}

Publish returned when SNS accepted the message. The queue got it through the raw subscription, the adapter received it with a one-second long poll, and Ack deleted it.

5. Make it fail

Change MaxDeliver to 3 and run again. Start fails with the adapter's refusal inside it:

messaging: subscription "billing" could not start: messaging: subscribing "billing": messaging-sqs: MaxDeliver 3 but queue "billing"'s maxReceiveCount is 5; the two must be equal, because the queue's redrive is where the budget is enforced

The queue's redrive policy is where the retry budget is enforced, so the subscription has to agree with it. Put MaxDeliver back to 5.

Then publish something no format can read, straight to the topic, past the adapter:

$ aws sns publish --topic-arn $TOPIC --message 'not an event'

Run the program. The handler never sees it. The adapter logs that a message could not be read, each of its five receives releases it at once, and SQS then moves it to billing-dlq. See What happens to a message nothing can read.

Clean up

$ docker stop sqs-tutorial

Where next

Build the topology for a real account, including FIFO and the queue policy AWS enforces and LocalStack does not. Connect to AWS for credentials and region.