View source for User:Crazycomputers/WatchlistBot 2

This page documents the progress towards the next major version of WatchlistBot.  This version is a complete rewrite.

== Changes ==
=== Runs on Node ===
I've decided to write this version in JavaScript, using Node as the runtime.  I find JavaScript much easier to work than C# these days, and npm is a much more vibrant ecosystem than NuGet.  Very little of the existing code would be useful anyway, as nearly all aspects of the bot's architecture are changing.

=== New features ===
* A web application will be the primary interface for the bot.  Changes that match a watchlist entry will be delivered to this interface in real-time, with support for desktop notifications.
* Matched changes will be recorded for later review, regardless of whether the web application is open at the time of a change.
* Each user can optionally enable one notification mechanism for real-time notification of matched changes to a third-party service outside of the web application.
** Each watchlist entry can be individually configured to trigger these notifications.  For example, a user could turn on this setting for only high-priority changes.
** Failed notification attempts will be retried for some amount of time.
* Watchlist entries can be individually configured to expire.
* Each entry can optionally be assigned a category, which is used to group recorded changes in the web UI.
* REST API for watchlist manipulation.  WatchlistBot 2 could be easily integrated with counter-vandalism tools, for example.

=== Multi-service architecture ===
All major components run as separate containerized services that communicate using a message broker.  This enables each component to be individually restarted for upgrades without affecting the other components, and more generally gives all of the benefits of loose coupling.  Each component is written so that it can be restarted without missing any messages from other components.  These are the primary components:

* Wikimedia event stream bridge ({{done}}): forwards changes from the Wikimedia event stream to a local queue.
* Change monitor ({{done}}): consumes changes from the local queue and compares them to watchlist entries.  On a match, records a change in the database and notifies the web UI (via a message exchange) of the change.  If any matched watchlist entries are configured to notify a third-party service of the change, publishes a message to the appropriate service-specific notification queue.
* Notification agent ({{semi-done}} -- see below for service list): consumes messages from service-specific notification queues and delivers notifications to those services.
* Web UI ({{done}}): REST API server providing the web UI, including watchlist manipulation, desktop notifications, and a real-time event stream of matched changes.

== Notification services ==
It is still occasionally useful to be notified via mechanisms other than the web UI.  Perhaps you want to be notified of certain changes on your mobile device so you can be aware of them when you are not at your computer.

The following notification services will be supported:

* [[Webhook]]s.  This will support many different services with very little effort.  Many such services have a desktop and mobile app that can receive push notifications.  Some services may require minor changes to the webhook payload, as not all accept the same payload structure.
** [[Discord (software)|Discord]] ({{done}})
** [[Slack (software)|Slack]] ({{notdone}})
** [https://rocket.chat/ rocket.chat] ({{notdone}})
** [https://www.pushbullet.com/ Pushbullet] ({{not done not likely}}): Requires a paid account to send more than 500 notifications in a month. May not be supported because of this restriction.
* [[Matrix (protocol)|Matrix]] {{notdone}}

The following services may be supported, but this is not decided yet:

* [[XMPP]]: This was the only channel supported by the original. Interest in XMPP appears to be mostly gone at this point, though I do not want to abandon my users who are still using it. Note that this version of the bot would not accept commands over XMPP; from the bot's perspective, notification services are send-only.

The following channels will likely not be supported:

* [[Internet Relay Chat|IRC]]: This would likely be restricted to just one or two servers.  An authentication mechanism would have to be supported since IRC's protection against impersonation is a bit relaxed, even when enabled -- we don't want to allow one user to receive another user's notifications.  Put simply, it's a lot of work for a likely-small user base.

I am open to suggestions for other services.