UnrealIRCd 5.0.0-alpha3 released

News about the UnrealIRCd project, including release announcements
Post Reply
Syzop
UnrealIRCd head coder
Posts: 1890
Joined: Sat Mar 06, 2004 8:57 pm
Location: .nl
Contact:

UnrealIRCd 5.0.0-alpha3 released

Post by Syzop » Sat Sep 21, 2019 4:44 pm

Hi everyone,

The third alpha version of UnrealIRCd 5 is now available for download. This is not a stable version, it is for testing only and the server may misbehave. Some planned functionality for UnrealIRCd 5 is still missing, but about 90% is there. UnrealIRCd 5 alpha versions are there to give users an early opportunity to see what the current state of UnrealIRCd 5 is. For those users who do dare to run it, feel free to report any issues you may find or comment on the many U5 features on https://bugs.unrealircd.org/.

Most people will only want to read the release notes, rather than run or even test this version. That's perfectly fine. Be prepared for a long read.

Changes between -alpha2 and -alpha3
First, a short summary of the changes between 5.0.0-alpha1 and 5.0.0-alpha2:
  • Lots of code cleanups again and things that break modules
  • To protect against evading spamfilters, spamfilter now strips UTF8 nonbreakable spaces from input.
  • ./Config asks fewer questions (this was already in alpha2) and the configure process is faster, since we assume you are using a modern OS.
  • The floodprot module was kicking innocent users if 't' (text flood) was in use without 'r' (repeat)
  • New set::min-nick-length to set a minimum nick length
  • New require module { }, deny module { } to require or deny certain modules on UnrealIRCd 5 servers, otherwise the link is rejected. See also set::require-module for some related settings.
  • Conditional configuration was not working properly
  • WATCH was not working because the numeric output was wrong due to a faulty mass replace earlier in U5, same with ADMIN.
  • Ban exceptions (ELINE) were not stored properly in tkl.db
  • More crash reporter improvements
  • We are also in the process of removing compatibility code for old protocols, such as UnrealIRCd 3.2.x. This will further clean up the source code and will get rid of old bugs.
Release notes

Summary:
The most visible change to end-users is channel history. Various modules from Gottem have been integrated and enhanced.
Channel settings of +P channels and *LINES are saved in a database and restored on startup (via 'channeldb' and 'tkldb' respectively).
Channel mode +L has a slight change of meaning, the existing floodprot mode (+f) has a new type to prevent repeated messages and a new drop action. A few extended bans have been added as well (~f and ~p).
IRCOps now have the ability to add ban exceptions via the /ELINE command.
Advanced admins can use more dynamic configuration options where you can define variables and use them later in the configuration file.
Finally, there have been speed improvements, we use better defaults and have added more countermeasures and options against spambots.
Under the hood a lot of the source code was changed and cleaned up.

Enhancements:
  • Support for server generated message tags, which allows us to communicate additional information in protocol messages such as in JOIN and PRIVMSG. Currently implemented and permitted message tags are:
    • account: communicate the services account that a user uses
    • msgid: assign an unique message id to each message
    • time: assign a time label to each message
    The last two are mainly for history playback.
  • Support for IRCv3 "echo-message", which helps clients, among other things, to see if the message you sent was altered in any way, eg: censored, stripped from color, etc.
  • Support for IRCv3 "BATCH", needed for some other features.
  • Recording and playback of channel history when channel mode +H is set.
    The syntax is: +H max-lines-to-record:max-time-to-record-in-minutes.
    For example: +H 50:1440 means the last 50 messages will be stored and no message will be stored longer than 1440 minutes (1 day).
    The channel history is then played back when joining such a channel, but with two things to keep in mind:
    1. The client must support the 'server-time' CAP, otherwise history is not shown. Any modern IRC client supports this.
    2. Only a maximum of 15 lines are played back on-join by default
    The reason for the maximum 15 lines on-join playback is that this can be quite annoying if you rejoin repeatedly and as to not flood the users screen too much (unwanted). In the future we will support a mechanism for clients to "fetch" history - rather than sending it on-join - so they can fetch more than the 15 lines, up to the number of lines and time configured in the +H channel mode.
    You can configure the exact number of lines that are played back and all the limits that apply to +H via set::history::channel.
  • For saving and retrieving history we currently have the following options:
    • history_backend_mem: channel history is stored in memory. This is very fast but also means history is lost on restart.
    • history_backend_null: don't store channel history at all. This can be useful to load on servers with no users on it, such as a hub server, where storing history is unnecessary.
    As you can see there is currently no 'disk' backend. However, in the future more options may be added.
    Also note that 3rd party modules can add history backends as well.
  • Support for ban exceptions via the new /ELINE command. This allows you
    to add exceptions for regular bans (KLINE/GLINE/ZLINE/etc), but also
    for connection throttling and blacklist checking.
    For more information, just type /ELINE in your IRC client as an IRCOp.
  • Channel mode +L now kicks in for any rejected join, so not just for +l but also for +b, +i, +O, +z, +R and +k.
    If, for example, the channel is +L #insecure and also +z then, when an insecure user ties to join, they will be redirected to #insecure.
  • New extended ban ~f to forward users to the specified channel if the ban matches. Example: +b ~f:#badisp:*!*@*.isp.org
  • Channel mode +f now has a 'd' action: drop message. This will send an error message to the user and not show the message in the channel but otherwise do nothing (no kick or ban).
    Example: +f [5t#d]:15 will limit sending a maximum of 5 messages per 15 seconds per-user and drop any messages sent above that limit.
  • Channel mode +f now has 'r' floodtype to prevent repeated lines. This will compare the current message to the last message and the one before that the user sent to the channel. If it's a repeat then the user can be kicked (the default action), the message can be dropped ('d') or the user can be banned ('b'). Example: +f [1r#d]:15. If you want to permit 1 repeated line but not 2 then use: +f [2r#d]:15
  • New module tkldb (loaded by default): all *LINES and spamfilters are now saved across reboots. No need for services for that anymore.
  • New module channeldb (loaded by default): saves and restores all channel settings including topic, modes, bans etc. of +P (persistent) channels.
  • New module restrict-commands, which allows you to restrict any IRC command based on criteria such as "how long is this user connected", "is this user registered (has a services account)" etc.
    The example.conf now ships with configuration to disable LIST the first 60 seconds and disable INVITE the first 120 seconds.
    If you are having spambot problems then tweaking this configuration may be helpful to you. See set::restrict-commands.
  • New options deny module, require module to deny and require modules on other UnrealIRCd 5 servers, otherwise the link is rejected. See also set::require-module.
  • New module rmtkl (loaded by default): this allows you to remove TKL's such as GLINEs easily via the /RMTKL command.
  • The reputation and connthrottle modules are now loaded by default.
    Just as a reminder, what these do is classifying your users in "known users (known IP's)" and "unknown IP's" for IP's that have not been seen before (or only for a short amount of time). Then, when there is a connection flood, unknown/new IP addresses are throttled at 20 connections per minute, while known users are always allowed in. For more information, see https://www.unrealircd.org/docs/Connthrottle
  • Add support for more dynamic configuration via @define and @if. This is mostly for power users, in particular users who share the same configuration file across several servers. New features are:
    • You can define variables, like:

      Code: Select all

      @define $SERVER "hub.example.org"
      ..and then use $SERVER anywhere in the configuration file such as:

      Code: Select all

      me {
              name "$SERVER";
              [..]
      }
    • You can have conditional configuration as well:

      Code: Select all

      @if $SERVER == "hub.example.org"
      link {
              [..]
      }
      @endif
    • In @if you can use module-loaded() to have conditional configuration based on if a module is loaded or not.
      For example, we have the following code in help.conf for HELPOP CHMODES so the line is only displayed if the chanmodes/noctcp module is loaded:

      Code: Select all

      helpop chmodes {
              [..]
      @if module-loaded("chanmodes/noctcp")
              " C = No CTCPs allowed in the channel [h]";
      @endif
      }
  • New extban ~p (+b ~p:*!*@*.nl) to hide PART and QUIT messages.
  • You will now see a warning when a server is not responding even before they time out.
    How long to wait for a PONG reply upon PING can be changed via set::ping-warning and defaults to 15 seconds.
    If you see the warning frequently then your connection is flakey.
  • Add new setting set::broadcast-channel-messages which defines when channel messages are sent across server links.
    The default setting is 'auto' which is the correct setting for pretty much everyone.
  • Add new option set::part-instead-of-quit-on-comment-change:
    when a QUIT message is changed due to channel restrictions, such as stripping color or censoring a word, we normally change the QUIT message. This has an effect on ALL channels, not just the one that imposed the restrictions. While we feel that is the best tradeoff, there is now also this new option (off by default) that will change the QUIT into a PART in such a case, so the other channels that do not have the restrictions (eg: are -S and -G) can still see the original QUIT message.
  • New module webredir. Quite some people run their IRCd on port 443 or 80 so their users can avoid firewall restrictions in place. In such a case, with this module, you can now send a HTTP redirect in case some user enters your IRC server name in their browser. Eg https://irc.example.org/ can be made to redirect to https://www.example.org/
    See https://www.unrealircd.org/docs/Set_blo ... redir::url
Changed:
  • The "except tkl" block is now called "except ban". If no type is specified in an except ban { } block then we exempt the entry from kline, gline, zline, gzline and shun.
  • Windows: we now require a 64-bit version, Windows 7 or later. The new program path is: C:\Program Files\UnrealIRCd 5 and the binaries have been moved to a new subdirectory: bin\
  • Modules lost their m_ prefix, so for example m_map is now just map. Also the modules in cap/ are now directly in modules.
  • More modules that were previously PERM (permanent) can now be unloaded and reloaded on the fly. This allows more "hotfixing" without restart in case of a bug and also more control for admins at runtime. Only <5 modules out of 173 are permanent now.
  • User mode +T now blocks channel CTCPs as well.
  • set::modes-on-join is now +nt by default.
  • The authprompt module is now loaded by default. This means that if you do a soft kline on someone (eg: KLINE %*@*.badisp) then the user has a chance to authenticate to services, even without SASL, and bypass the ban if (s)he is authenticated.
  • The WHOX module is now used by default. Previously it was optional.
    WHOX enhances the "WHO" output, providing additional information to IRC clients such as the services account that someone is using.
    It is more universal than standard WHO. Unfortunately this also means the WHO syntax changed to something less logical.
  • At many places the term "SSL" has been changed to "SSL/TLS" or "TLS".
    Configuration items (eg: set::ssl to set::tls) have been renamed as well and so have directories (eg: conf/ssl to conf/tls).
    The old configuration names still work and currently does NOT raise any warning. Also, when upgrading an existing installation on *NIX, the conf/tls directory will be symlinked to conf/ssl as to not break any Let's Encrypt certificate scripts.
  • It is now mandatory to have at least one open SSL/TLS port, otherwise UnrealIRCd will refuse to boot. Previously this was a warning.
  • IRCOps now need to use SSL/TLS in order to oper up, as the set::plaintext-policy::oper default setting is now 'deny'.
  • Both set::outdated-tls-policy::oper and set::outdated-tls-policy::server are now 'deny', since all ircops and servers should use reasonable SSL/TLS protocols and ciphers.
  • The default generated certificated has been changed from RSA 4096 bits to Elliptic Curve Cryptography "384r1".
    This provides the same amount of security but at higher speed. This only affects the default self-signed certificate. You can still use RSA certificates just fine.
  • If you do use an RSA certificate, we now require it to be at least 2048 bits otherwise UnrealIRCd will refuse to boot.
  • When matching allow { } blocks, we now always continue with the next block (if any) if the password did not match or no password was specified. In other words, allow::options::nopasscont is now the default and we behave as if there was a ::wrongpasscont too.
  • All snomasks are now oper-only. Previously some were not, which was confusing and could lead to information leaks. Also removed weird set::snomask-on-connect accordingly.
  • The IRCd now uses hash tables that are resilient against hash table attacks. Also, the hash tables have increased in size to speed things up when looking up nick names etc.
  • Server options in VERSION (eg: Fhin6OoEMR3) are no longer shown to normal users. They don't mean much nowadays anyway.
  • We now default to system libs (eg: --with-system-pcre2 is assumed)
  • ./Config now asks fewer questions and configure runs faster since many unnecessary checks have been removed (compatibility with very old compilers / systems).
  • Spamfilter should catch some more spam evasion techniques.
Minor issues fixed:
  • Specifying a custom OpenSSL/LibreSSL path works now
Removed:
  • Extended ban ~R (registered nick): this was the old method to match registered users. Everyone should use ~a (services account) instead.
  • The old TRE 'posix' regex method has been removed because the TRE library is no longer maintained for over a decade and contains many bugs. (It was already deprecated in UnrealIRCd 4.2.3). Use type 'regex' instead, which uses the modern PCRE2 regex engine.
  • Timesync support has been removed. Use your OS time synchronization instead. (Note that Timesync was already disabled by default in 2018)
  • Changing time offsets via TSCTL OFFSET and TSCTL SVSTIME are no longer supported. Use your OS time synchronization (NTP!). Adjustments via TSCTL are simply not accurate enough.
  • The 'nopost' module was removed since it no longer serves any useful purpose. UnrealIRCd already protects against these kind of attacks via ping cookies (set::ping-cookie, enabled by default).
Developers:
IMPORTANT: As long as UnrealIRCd is in alpha, we do not suggest 3rd party module authors to start porting your modules yet. Of course you may, but the module API is still very likely to change, so you may have to do certain (other) changes again next alpha release. Best to wait until beta1. You have been warned ;)
  • The module header is now as follows:

    Code: Select all

      ModuleHeader MOD_HEADER
        = {
              "nameofmodule",
              "5.0",
              "Some description", 
              "Name of Author",
              "unrealircd-5",
          };
    The changes are:
    • The name of the module must match the loadmodule name. So for example third/funmod must also be named third/funmod.
    • The version must start with a digit and may only contain a-z 0-9 and - and .
    • The description is still free text
    • There's a new author field
    • The struct ends with "unrealircd-5" to signal UnrealIRCd 5 compatibility
  • The MOD_TEST, MOD_INIT, MOD_LOAD and MOD_UNLOAD functions no longer take a name argument. So: MOD_INIT(mymod) is now MOD_INIT()
  • In UnrealIRCd 5, during development, --enable-asan is ON by default to catch more bugs. This also means an up to 10x slowdown and more memory usage. When we reach 5.0.0 stable this will be turned off.
  • We now use our own BuildBot infrastructure, so Travis-CI and AppVeyor have been removed.
  • We now use a new test framework.
  • Auth_Check() now returns 1 for allow and 0 on deny (!!)
  • New function new_message() which should be called when a new message is sent, or at least for all channel events. It adds (or inherits) message tags like 'account', 'msgid', 'time', etc.
  • Many send functions now take an extra MessageTag *mtags parameter, including but not limited to: sendto_one() and sendto_server().
  • New single unified sendto_channel() and sendto_local_common_channels() functions that are used by all the channel commands.
  • Numerics should now be sent using sendnumeric(). There's also a format string version sendnumericfmt() in case you need it.
  • The parameters in several hooks have changed. Many now have an extra MessageTag *mtags parameter. Sometimes there are other changes as well, for example HOOKTYPE_CHANMSG now has 4 extra parameters.
  • If you ever send a timestamp in a printf-like function, such as in sendto_server(), then be sure to use %lld and cast the timestamp to 'long long' so that it is compatible with both *NIX and Windows.
    Example: sendnotice(sptr, "Timestamp is %lld", (long long)ts);
  • New HOOKTYPE_IS_HANDSHAKE_FINISHED. If a module returns 0 there, then the register_user() function will not be called and the user will
    not come online (yet). This is used by CAP and some other stuff. Can be useful if your module needs to "hold" a user in the registration phase.
  • The function is_module_loaded() now takes a relative path like "usermodes/noctcp" because with just "ctcp" one could not see the difference between usermodes/noctcp and chanmodes/noctcp.
  • CHFL_CHANPROT is now CHFL_CHANADMIN, is_chanprot() is now is_chanadmin()
  • All hash tables now use SipHash, which is a hash function that is resilient against hash table attacks. If you, as a module dev, too use any hash tables anywhere (note: this is quite rare), then you are recommended to use our functions, see the functions siphash() and siphash_nocase() in src/hash.c.
  • The random generator has been updated to use ChaCha (more modern).
  • You can now save pointers and integers etc. across rehashes by using LoadPersistentPointer() and SavePersistentPointer().
    For an example, see src/modules/chanmodes/floodprot.c how this can be used. Note that there can be no struct or type changes between rehashes.
  • New ModData types: MODDATA_LOCALVAR and MODDA_GLOBALVAR. These are settings or things that are locally or globally identified by the variable name only and not attached to any user/channel.
  • Various files have been renamed. As previously mentioned, the m_ prefix was dropped in src/modules/m_*.c. Similarly the s_ prefix was dropped in src/s_*.c since it no longer had meaning. Also some files have been deleted and integrated elsewhere or renamed to have a name that better reflects their true meaning.
Server protocol:
  • Surprisingly little has been changed in the server to server protocol between UnrealIRCd 4 and UnrealIRCd 5:
  • "PROTOCTL MTAGS" indicates that the server is capable of handling message tags and that the server can cope with 4K lines. (Note that the ordinary non-message-tag part is still limited to 512 bytes).
  • We now assume at least UnrealIRCd 4 like protocol abilities, such as SID, NICKv2, TKLEXT2, etc. More precise information will follow.
    In particular this means UnrealIRCd 3.2.x will not link with 5.x.

Post Reply