Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,11 @@ and how they interact with the message store, message index and so on.
### Variable Queue Guide ###

Ultimately, messages end up queued at the
[backing queue](https://github.com/rabbitmq/rabbitmq-common/blob/master/src/rabbit_backing_queue.erl). From
[backing queue](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_backing_queue.erl). From
here they can be retrieved, acked, purged, and so on. The most common
implementation of the backing queue behaviour is the
`rabbit_variable_queue`
[module](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_variable_queue.erl),
[module](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_variable_queue.erl),
explained in the following guide:

[Variable Queue](./variable_queue.md)
Expand Down
4 changes: 2 additions & 2 deletions authorization_and_authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ In practice in 99% of cases only two mechanisms are used:

* `PLAIN` (a set of credentials such as username and password)
* `EXTERNAL`, which assumes authentication happens out of band (not performed
by RabbitMQ authN backends), usually [using x509 (TLS) certificates](https://github.com/rabbitmq/rabbitmq-auth-mechanism-ssl).
by RabbitMQ authN backends), usually [using x509 (TLS) certificates](https://github.com/rabbitmq/rabbitmq-server/tree/master/deps/rabbitmq_auth_mechanism_ssl).
This mechanism ignores client-provided credentials and relies on TLS [peer certificate chain
verification](https://tools.ietf.org/html/rfc6818).

When a client connection reaches [authentication stage](https://github.com/rabbitmq/rabbitmq-server/blob/v3.7.2/src/rabbit_reader.erl#L1304), a mechanism requested by the client
and supported by the server is selected. The mechanism module then checks whether it can
be applied to a connection (e.g. the TLS-based mechanism will reject non-TLS connections).

An authentication mechanism is a module that implements the [rabbit_auth_mechanism](https://github.com/rabbitmq/rabbitmq-common/blob/master/src/rabbit_auth_mechanism.erl) behaviour, which includes
An authentication mechanism is a module that implements the [rabbit_auth_mechanism](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit_common/src/rabbit_auth_mechanism.erl) behaviour, which includes
3 functions:

* `init/1`: self-explanatory
Expand Down
2 changes: 1 addition & 1 deletion basic_publish.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ called `"my_exchange"`, we will end up with the following record:

Resources like that one are used everywhere in RabbitMQ, so it's a
good idea to study their parts in the
[rabbit_types](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_types.erl)
[rabbit_types](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit_common/src/rabbit_types.erl)
module where this declarations are defined.

Once we have the exchange record, `basic.publish` will use it to see
Expand Down
6 changes: 3 additions & 3 deletions deliver_to_queues.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ via the `delegate` framework. The Erlang message will have this shape:
```

You can learn more about the delegate framework
[here](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/delegate.erl#L19).
[here](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit_common/src/delegate.erl#L10).

## AMQQueue Process Message Handling ##

Expand All @@ -51,7 +51,7 @@ callback will ack the `credit_flow` issued in above, and it will
monitor the message sender. The message sender is usually the
`rabbit_channel` that received the process. This pid is tracked using
the
[pmon module](https://github.com/rabbitmq/rabbitmq-common/blob/master/src/pmon.erl). The
[pmon module](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit_common/src/pmon.erl). The
state is kept as part of the `senders` field in the gen_server state
record. Once the message sender is accounted for the delivery is
passed to the function `deliver_or_enqueue/3`. There is where the
Expand Down Expand Up @@ -171,7 +171,7 @@ channel forever if a queue that's blocking it is actually down.

Take a look at the `handle_info` channel callback for the case when a
`DOWN` message is
[received](https://github.com/rabbitmq/rabbitmq-common/blob/master/src/rabbit_channel.erl#L578).
[received](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_channel.erl#L818).

### Process Mandatory Messages ###

Expand Down
6 changes: 3 additions & 3 deletions exchange_decorators.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ alone the whole routing logic to the underlying exchange.
Decorators are usually associated with exchanges via policies.

See the `active_for/1`
[callback](https://github.com/rabbitmq/rabbitmq-common/blob/master/src/rabbit_exchange_decorator.erl#L70)
[callback](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_exchange_decorator.erl#L70)
to understand which functions on the exchange would be decorated.

Take a look at the
[Sharding Plugin](https://github.com/rabbitmq/rabbitmq-sharding/blob/master/src/rabbit_sharding_exchange_decorator.erl)
[Sharding Plugin](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbitmq_sharding/src/rabbit_sharding_exchange_decorator.erl)
and the
[Federation Plugin](https://github.com/rabbitmq/rabbitmq-federation/blob/master/src/rabbit_federation_exchange.erl)
[Federation Plugin](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbitmq_federation/src/rabbit_federation_exchange.erl)
to see how exchange decorators are implemented.
4 changes: 2 additions & 2 deletions interceptors.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Interceptors are modules implemented as behaviours that allow plugin
authors to intercept and modify AMQP methods before they are handled
by the channel process. They were originally created for the
development of the
[Sharding Plugin](https://github.com/rabbitmq/rabbitmq-sharding/blob/master/README.extra.md#intercepted-channel-behaviour)
[Sharding Plugin](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbitmq_sharding/README.extra.md#intercepted-channel-behaviour)
to facilitate mapping queue names as specified by users vs. the actual
names used by sharded queues. Another plugin using interceptors is the
[Message Timestamp Plugin](https://github.com/rabbitmq/rabbitmq-message-timestamp)
Expand All @@ -16,7 +16,7 @@ behaviour. The most important callback is `intercept/3` where an
interceptor will be provided with the original AMQP method record that
the channel should process, the AMQP method content, if any, and the
interceptor state (see
[init/1](https://github.com/rabbitmq/rabbitmq-common/blob/master/src/rabbit_channel_interceptor.erl#L36)). This
[init/1](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_channel_interceptor.erl#L36)). This
callback should take the AMQP method that was passed to it, and the
content, and modify it accordingly. For example, the Sharding Plugin
will receive a `basic.consume` method, with a sharded queue called
Expand Down
2 changes: 1 addition & 1 deletion internal_events.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ dummy usernames are used, e.g. `rmq-internal` or `rmq-cli`.

## rabbitmq-event-exchange Plugin

[rabbitmq-event-exchange](https://github.com/rabbitmq/rabbitmq-event-exchange) is a plugin that consumes internal events
[rabbitmq-event-exchange](https://github.com/rabbitmq/rabbitmq-server/tree/master/deps/rabbitmq_event_exchange) is a plugin that consumes internal events
and re-publishes them to a topic exchange, thus exposing the events
to clients (applications).

Expand Down
6 changes: 5 additions & 1 deletion mandatory_message_handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,13 @@ see how the channel handles mandatory messages.

## Tracking Mandatory Messages ##

Note: Implementation was changed in
[3.8.0](https://github.com/rabbitmq/rabbitmq-server/pull/1831),
removing the dtree strucutre described below.

Mandatory messages are tracked in the `mandatory` field of the
channel's state record. Messages are tracked using our own
[dtree](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/dtree.erl)
[dtree](https://github.com/rabbitmq/rabbitmq-server/blob/v3.7.28/src/dtree.erl)
data structure. As explained in that module documentation, entries on
the _dual-index tree_ are stored using a primary key, a set of
secondary keys, and a value. In the case of tracking mandatory
Expand Down
10 changes: 5 additions & 5 deletions metrics_and_management_plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ There are four main components:
Core metrics are implemented in the rabbitmq server itself consisting of
a set of of ETS tables storing either counters or proplists containing details
or metrics of various entities. The schema of each table is documented in
[rabbit_core_metrics.hrl](https://github.com/rabbitmq/rabbitmq-common/blob/master/include/rabbit_core_metrics.hrl)
[rabbit_core_metrics.hrl](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit_common/include/rabbit_core_metrics.hrl)
in `rabbitmq-common`.

Mostly counters that are incremented in real-time as message interactions occur
Expand All @@ -45,15 +45,15 @@ memory overhead in relation to the number of active entities in the system.

## Management Agent

[rabbitmq-managment-agent](https://github.com/rabbitmq/rabbitmq-management-agent) is responsible for turning core metrics into
[rabbitmq-managment-agent](https://github.com/rabbitmq/rabbitmq-server/tree/master/deps/rabbitmq_management_agent) is responsible for turning core metrics into
data structures suitable for `rabbitmq-management` consumption. This is
done on a per node basis. There are no inter-node communications involved.

The management agent runs a set of metrics collector processes. There is one
process per core metrics table. Each collector periodically read its associated
core metrics table and performs some table-specific processing which produces
new data points to be inserted into the management metrics tables (defined in
[rabbitmq_mgmt_metrics.hrl](https://github.com/rabbitmq/rabbitmq-management-agent/blob/master/include/rabbit_mgmt_metrics.hrl)).
[rabbitmq_mgmt_metrics.hrl](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbitmq_management_agent/include/rabbit_mgmt_metrics.hrl)).
The collection interval is determined by the smallest configured retention intervals.

In addition to the collector processes there is a garbage collection event
Expand All @@ -76,7 +76,7 @@ This has no effect on the user but test suites that use the HTTP API would often

### exometer_slide

The [exometer_slide](https://github.com/rabbitmq/rabbitmq-management-agent/blob/master/src/exometer_slide.erl)
The [exometer_slide](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbitmq_management_agent/src/exometer_slide.erl)
module is a key part of the management stats processing.
It allows us to reasonably efficiently store a sliding window of incoming metrics
and also perform various processing on this window. It was extracted from the
Expand All @@ -93,7 +93,7 @@ The `rabbitmq-management` plugin is now mostly a fairly thin HTTP API layer.

It also handles the distributed querying and stats merging logic. When a stats
request comes in the plugin contacts each node in parallel for a set of "raw"
stats (typically `exometer_slide` instances). It uses the [delegate](https://github.com/rabbitmq/rabbitmq-common/blob/master/src/delegate.erl)
stats (typically `exometer_slide` instances). It uses the [delegate](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit_common/src/delegate.erl)
module for this and has it's own `delegate` supervision tree to avoid affecting
the one used for core rabbit delegations. Once stats for
each node has been collected it merges the data then proceeds with processing
Expand Down
6 changes: 5 additions & 1 deletion publisher_confirms.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ documentation in order to understand the feature.

## Tracking Confirms ##

Note: Implementation was slightly changed in
[3.8.0](https://github.com/rabbitmq/rabbitmq-server/pull/1893),
replacing the dtree strucutre described below.

Confirms work a bit differently than mandatory messages and
`basic.return`. In the case of mandatory messages we only need to send
a `basic.return` if the message can't be routed, but for publisher
Expand All @@ -19,7 +23,7 @@ fields in the channel's state record in order to track this
information.

The first one is a
[dtree](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/dtree.erl)
[dtree](https://github.com/rabbitmq/rabbitmq-server/blob/v3.7.28/src/dtree.erl)
stored in the field `unconfirmed`, which keeps track of the `MsgSeqNo`
associated with the QPids to which the message was delivered and
the Exchange Name used to publish the message. As explained in the
Expand Down
2 changes: 1 addition & 1 deletion queue_decorators.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ or by the federation plugin, to know when to move messages across
[federated queues](https://www.rabbitmq.com/federated-queues.html).

Decorators need to implement the `rabbit_queue_decorator`
[behaviour](https://github.com/rabbitmq/rabbitmq-common/blob/master/src/rabbit_queue_decorator.erl)
[behaviour](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_queue_decorator.erl)
and are usually associated with queues via policies.

A Queue decorator can receive notifications of the following events:
Expand Down
2 changes: 1 addition & 1 deletion queues_and_message_store.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ should be atomic.
Message store logic assumes that lookup operations for non-existent message
locations (if message is not yet written to file) are cheap.

See the [message store index behaviour module](https://github.com/rabbitmq/rabbitmq-common/blob/master/src/rabbit_msg_store_index.erl) for more details.
See the [message store index behaviour module](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit_common/src/rabbit_msg_store_index.erl) for more details.

The message store also needs to be garbage collected. There's an extra
process for GC (so that GC can lock some files and the message store
Expand Down
12 changes: 6 additions & 6 deletions transactions_in_exchange_modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,24 +19,24 @@ The question is, what's the purpose of that transaction parameter?
This is related to how RabbitMQ runs Mnesia transactions for its
internal bookkeeping:

[https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_misc.erl#L546](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_misc.erl#L546)
[rabbit_misc:execute_mnesia_transaction/2](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit_common/src/rabbit_misc.erl#L586)

As you can see in that code there's this PrePostCommitFun which is
called in Mnesia transaction context, and after the transaction has
run.

So here for example:
[https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_exchange.erl#L176](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_exchange.erl#L176)
So here for example: in
[rabbit_exchange:declare/7](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_exchange.erl#L143)
the create callback from the exchange is called inside a Mnesia
transaction, and outside of afterwards.

You can see this in action/understand the usefulness of it when
considering an exchange like the topic exchange which keeps track of
its own data structures:

[https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_exchange_type_topic.erl#L54](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_exchange_type_topic.erl#L54)
[https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_exchange_type_topic.erl#L64](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_exchange_type_topic.erl#L64)
[https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_exchange_type_topic.erl#L69](https://github.com/rabbitmq/rabbitmq-server/blob/master/src/rabbit_exchange_type_topic.erl#L69)
[rabbit_exchange_type_topic:delete/3](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_exchange_type_topic.erl#L49)
[rabbit_exchange_type_topic:add_binding/3](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_exchange_type_topic.erl#L59)
[rabbit_exchange_type_topic:remove_bindings/3](https://github.com/rabbitmq/rabbitmq-server/blob/master/deps/rabbit/src/rabbit_exchange_type_topic.erl#L64)

Deleting the exchange, adding or removing bindings, are all done
inside a Mnesia transaction for consistency reasons.