- ::
This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode
Firehose: A unified message bus for Infra services¶
https://storyboard.openstack.org/#!/story/2000643
Using MQTT start running a unified message bus that community infrastructure services can emit events on. This will enable efficient external consumption of infrastructure events for experimentation and rapid innovation of tooling within the community.
Problem Description¶
There are a lot of services running in openstack-infra doing lots of different things all the time. Following events generated by these services can be tricky, and we don’t have a consistent solution to making arbitrary state change details externally visible (so we fall back to relying on those with access to the machines letting us know what operations are being performed). What event data does exist is inconsistently presented in a variety of places over different protocols and requires significant effort to correlate cross-service.
Proposed Change¶
The goal of this spec is to create a unified message bus that Infra services can publish event streams to. This will enable external public consumption of Infra events in a consistent manner.
To accomplish this we will deploy a new server and run the mosquitto MQTT broker on it. MQTT protocol was designed in the late ’90s to be lightweight pub-sub protocol that would work with connections to remote locations that needed a “small code footprint” and where there was limited network bandwith. More recently this has made it quite popular in the IoT space. However, it is also a good fit for the distributed cloud model that a lot of Infra services use. We will also deploy mosquitto using websockets this should also provide a benefit to international community members where things like the Gerrit event stream are problematic to access over Gerrit’s SSH port.
That will provide the base infrastructure for publishing and subscribing to events but nothing will be using MQTT after that. To start the event reporting we’ll leverage germqtt (and make it an Infra project). This will be the initial usage for the unified message bus and make the Gerrit event stream available over MQTT. While the scope of this spec ends with getting Gerrit into MQTT, once we have an example we can more easily add more sources reporting directly into MQTT.
Alternatives¶
Use fedmsg¶
The fedora infrastructure team had a similar need/desire to have a unified message bus and they started the fedmsg project to address this need. The primary issue with fedmsg is that it depends on AMQP or ZMQ both of which we have had previous issues with. The choice to go with MQTT was partially a reaction to that.
Use a different mqtt broker¶
There are several open source MQTT brokers besides Mosquitto out there including ones like RabbitMQ, VerneMQ, and mosca.
Mosquitto was chosen because it’s a lightweight C program which will need less resources to run it. It also should be simpler to deploy since we don’t have to worry about deploying a JavaVM, nodejs, or erlang (which seems to be quite popular for writing message brokers).
Implementation¶
Assignee(s)¶
Primary assignee:
mtreinish
fungi
Gerrit Topic¶
Use Gerrit topic “firehose” for all patches related to this spec.
git-review -t firehose
Work Items¶
Write puppet-mosquitto module
- Deploy firehose01 server with puppet-mosquitto module and a
firehose.openstack.org CNAME to it
Add germqtt to openstack-infra
Write puppet-germqtt
Deploy puppet-germqtt on firehose01
Repositories¶
2 New repos will be created in the initial implemention, puppet-mosquitto and puppet-germqtt.
Servers¶
1 new server will need to be created for running mosquitto, firehose01. Since mosquitto and MQTT is fairly lightweight we probably can get away with a smaller server (2GB flavor should be plenty).
DNS Entries¶
Forward/reverse DNS entries will be added for firehose01.openstack.org, as well as a CNAME of firehose.openstack.org.
Documentation¶
Initial service documentation will be added to the openstack-infra/system-config repo as usual. An availability announcement will be sent to the openstack-dev mailing list.
Security¶
Sensitive data should not be published over the message bus. It is public and anonymously accessible.
Testing¶
The new Puppet modules will run our usual Puppet test and style jobs, and the germqtt repo will have typical Python unit testing and style checks as well.
Dependencies¶
This will be the first use of germqtt in infra (which will be added as an infra project)
2 new puppet modules will be created, 1 to deploy mosquitto and the other to run germqtt
