/ home / blog about

Wake-on-LAN and Push Notifications in Mobile Linux

2023/12/09

the more i carry my Pinephone for long stretches, the more i feel limited by battery life. there's a lot one can do to deal with that, and among those options is to reduce idle power draw.

if you use a desktop environment like SXMO, it'll keep a charge for the whole day if you were to keep it screen-off in your pocket. it does that by suspending the CPU to RAM, but keeping the modem powered -- great if all you care about is being notified on incoming calls/texts, not so great if you want to be notified by apps running on the CPU.

a typical solution to this is to wake the CPU whenever the system gets an IP packet, yield that to userspace & give your applications a chance to sound a ringer, vibrate, etc, and then suspend again when they're done.

the more you can decide "this packet isn't urgent" without CPU intervention, the more you can avoid waking the CPU, the more you can extend battery life. and so most systems provide tooling to help you tune IP-based wakeups if you know where to poke.

Wake on LAN

a.k.a. "Wake-On-Wireless LAN" or "wowlan". Wake on LAN is an actual established standard for remotely waking one ethernet device from another device on the same LAN. the user or OS enables this feature in the BIOS, the PC suspends, some other device sends a specially crafted packet (a "magic packet"), magic happens, and the BIOS wakes the system.

this can work on WiFi, too, but on a mobile system you enable this by speaking directly to the WiFi chip, rather than BIOS/UEFI. if you're lucky, sudo iw phy0 wowlan enable any will do just that.

then enter sleep (rtcwake -m mem -s 300 to suspend to RAM for 300 seconds), and from a different device on the same LAN, wake your phone with wol <your_phones_mac_address> e.g. wol 02:ba:7c:9c:cc:78 (find the MAC address with iw dev).

for the Pinephone, whether or not this works depends on how your kernel is configured. firstly, your kernel needs to know to keep the WiFi chip powered during suspend. second, there's a GPIO routed from the WiFi chip to the main SoC: the WiFi chip toggles this GPIO when it sees the magic packet, and the SoC needs to be told to recognize that as an interrupt source that the kernel can respond to. megi's kernel does these things, i can't say about other kernels like Mobian's.

so if you're on megi's kernel, this should work... at least once if you repeat the process enough. WiFi and Bluetooth share a lot of the same resources and don't always play nicely together. a quick to get WoL to work reliably is to just disable bluetooth: rm /lib/firmware/rtl_bt (back it up first). there's also a CONFIG_BT_COEXIST kernel option you could mess with if you're pursuing a longer-term fix (i don't really use bluetooth, but would appreciate to hear better fixes if anyone pursues them).

and now, wol should pretty reliably wake the phone. there's a race condition if you try that during the rtcwake call instead of waiting a second for it to complete -- i'll address that further down.

Wowlan for Userspace

you're probably not interested in manually calling wol from some other computer on the same network to wake your phone. you'd probably prefer it to wake anytime you get a Matrix or XMPP message, or something. for that, things diverge quickly from any sort of standard. but wol actually just sends out an ordinary UDP packet with a specific payload, so getting from there to "wake on TCP traffic sent to port 22", or "wake on TCP traffic sent from IP foo" shouldn't be technically difficult.

the Pinephone's Realtek WiFi chip exposes a programmable pattern-matching facility. just give it a byte string and a mask, and it'll wake on any packet whose bytes within the masked regions match those within the byte string. for example, IPv4 TCP packets specify their destination port at bytes 37-38. it's easy to tell the WiFi chip to "wake on any TCP packet sent to port 22". run this command before the rtcwake call, then try sshing into your phone while it's asleep:

sudo iwpriv wlan0 wow_set_pattern pattern=-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:00:16

with any luck, it should wake!

the fields there are hexadecimal, : indicates byte boundaries, - indicates "i don't care what value this byte is set to". but figuring out the byte pattern which achieves what you want is painful, so i've got a script for that here (mirrored). equivalent to the above is rtl8723cs-wowlan tcp --dest-port 22. if you have a chat application which is talking over https, you might try rtl8723cs-wowlan tcp --source-port 443 and see that the phone wakes every time you get a message. but then you'll get spurious wakes if you leave a web browser open, etc. you could use lsof -i4 to locate the local port(s) your IM app is using and wake more specifically on those. but you'll still hit limits with this approach, especially if you're using a chatty protocol like Matrix, or if you idle in a bunch of channels configured to notify you only on @mention.

enter... notification servers!

Notification Servers

when an application wants to sound a notification on iOS or Android when it's not in focus, it's not the application running on the phone which does that, but actually some other server (operated by or for the developer) which tells Apple's server (or Google's) about the notification, and then they relay it to your phone. one of the justification for that is precisely to allow the type of power-saving we're aiming for here, but because that approach poses privacy concerns, passionate people have created open source alternatives to the official Apple/Google notification delivery servers.

UnifiedPush defines the standards for each confusingly-named portion of this data flow, and ntfy provides hosted and self-hosted implementations for all the major components. most mature applications provide a way for you to integrate all this: i'll show how to do this for Matrix-synapse and Prosody (XMPP). both can be configured server-side if you have access to that, or client-side if your client exposes an option for it.

but first, let's prove this push notification system in a simpler CLI workflow.

Minimal ntfy Workflow

ntfy operates a free-to-use server for the push gateway component (i.e. the part that's normally routed through Apple/Google), so install the CLI package for the publish/subscribe portions and we can test a wake-on-notification setup trivially.

on your phone:

$ ntfy sub TEST_WOWLAN_TOPIC

then determine the local port that ntfy process connected from with lsof -i4 -P, and create a wake condition for it:

# clear out previous rules
$ rtl8723cs-wowlan enable-clean
$ rtl8723cs-wowlan tcp --dest-port 1234
#^ replace 1234 with the port from `lsof`

then sleep the phone with rtcwake -m mem, and from another computer (or the ntfy web interface) send a notification:

$ ntfy pub TEST_WOWLAN_TOPIC "hello phone"

if all is well, your phone should awake and the ntfy sub command from earlier should have printed "hello phone"!

Hosting a ntfy instance

this part is optional. most applications which speak UnifiedPush provide an option to only send a summary (like "new Matrix message") instead of the actual contents, so you only really need to concern yourself with this if you want the control of self-hosting or are worried more specifically about metadata leaks.

NixOS ships options for hosting your own ntfy. use it like:

services.ntfy-sh.enable = true;

ntfy listens on port 2586 by default, so you can subscribe like ntfy sub MY_INSTANCE_ADDRESS:2586/TEST_WOWLAN_TOPIC, and use a wowlan rule that matches the source port instead of checking lsof manually: rtl8723cs-wowlan tcp --source-port 2586.

however, running it this way allows anyone to use your server. the easy way to overcome this is to treat the topic as a shared secret, and forbid use of any topic except your secret topic. naturally, that only works if you secure the connection so do all this behind a TLS-capable reverse proxy like nginx:

services.ntfy-sh.enable = true;

services.nginx.virtualHosts."MY.NTFY.HOST" = {
  forceSSL = true;
  listen = [
    { addr = "0.0.0.0"; port = 2587; ssl = true; }
    { addr = "0.0.0.0"; port = 443;  ssl = true; }
  ];
  locations."/" = {
    proxyPass = "http://127.0.0.1:2586";
    proxyWebsockets = true;  #< support websocket upgrades. without that, `ntfy sub` hangs silently
    recommendedProxySettings = true; #< adds headers so ntfy logs include the real IP
    extraConfig = ''
      # absurdly long timeout (86400s=24h) so that we never hang up on clients.
      proxy_read_timeout 86400s;
    '';
  };
};

services.ntfy-sh.settings = {
  base-url = "https://MY.NTFY.HOST";
  behind-proxy = true;
  auth-default-access = "deny-all";
};
systemd.services.ntfy-sh.preStart = ''
  # note that this will fail upon first run, i.e. before ntfy has created its db.
  # just restart the service.
  ${pkgs.ntfy-sh}/bin/ntfy access everyone TEST_WOWLAN_TOPIC read-write
'';

deploy that and subscribe to the https url this time. note the port change to 2587: you still want a unique port against which you can write a wowlan rule, and it's easier to put nginx on a new port than ntfy's default 2586.

if you're thorough, you might notice some spurious wakeups with this setup. ntfy sends keep-alive packets every 45 seconds, but that's no good for us! best is to decrease/disable those keep-alives. i'll revisit the topic of keep-alives & sleep durations near the end of this article.

services.ntfy-sh.settings.keepalive-interval = "30m";

Synapse (Matrix) ntfy integration

the Synapse Matrix server exposes an API for controlling its push behavior here. some of the larger clients, like Fluffychat, can be seen to host their own push gateways, which they point Synapse to via this API. presumably one could configure these clients to request a different push gateway (i.e. ntfy.sh, or your own instance from just above), either in the UI or with an edit to their source code. but if you have CLI admin access to Synapse, it's easier to do this generically from the CLI.

first, grab an auth token for your Matrix account. it looks like 6cC_a03Ty3sTfqvo3FS_x8vEjdvNsxyL5W4mm73 and can be found in Element's "Help & About" settings page.

then, with this token, open a CLI to wherever you host your synapse server and issue this command to see where it's currently sending notifications (if anywhere):

$  curl --header "Authorization: Bearer ACCESS_TOKEN" \
  localhost:8008/_matrix/client/v3/pushers \
  | jq .
{
  "pushers": [
    {
      "app_display_name": "Element (iOS)",
      "app_id": "im.vector.app.ios.prod",
      "data": {
        "url": "https://matrix.org/_matrix/push/v1/notify",
        "format": "event_id_only",
        "default_payload": {
          "aps": {
            "mutable-content": 1,
            "alert": {
              "loc-key": "Notification",
              "loc-args": []
            }
          }
        }
      },
      "device_display_name": "iPhone",
      "kind": "http",
      "lang": "en-US",
      "profile_tag": "1Akvq5DDr159ANj9",
      "pushkey": "Bap1n7kqGzF9TuPkiDNDy9wW+cuvfDnxy7Cab7AMsIX="
    }
  ]
}

now, add a new pusher for your ntfy service. this won't replace the existing settings, instead it'll cause push notifications to be sent to two locations simultaneously:

$ curl --header "Authorization: Bearer ACCESS_TOKEN" \
  --data '{ \
    "app_display_name": "ntfy-adapter", \
    "app_id": "ntfy.uninsane.org", \
    "data": { \
      "url": "https://MY.NTFY.HOST/_matrix/push/v1/notify", \
      "format": "event_id_only" \
    }, \
    "device_display_name": "ntfy-adapter", \
    "kind": "http", \
    "lang": "en-US", \
    "profile_tag": "", \
    "pushkey": "TEST_WOWLAN_TOPIC" \
  }' \
  localhost:8008/_matrix/client/v3/pushers/set

repeat the first query and you'll see both of these listed. to delete the new pusher, repeat the above curl command with kind set to null.

anyway, put your phone to sleep, have someone send you a message that Synapse would alert you on, and now your phone should awake!

Prosody (XMPP) ntfy integration

Prosody has mod_cloud_notify and XEP-0357, but at the time of writing, client support is wanting. easier is to hack it in server-side.

Prosody has a Lua-based module system, so we can just author our own mod_ntfy_push (available here or here), drop it in the modules directory, and then import it:

services.prosody.extraPluginsPath = [ ./folder/containing/mod_ntfy_push ];
services.prosody.extraModules = [ "ntfy_push" ];
services.prosody.extraConfig = ''
  ntfy_endpoint = "https://MY.NTFY.HOST/TEST_WOWLAN_TOPIC"
'';

i've only authored this module to alert me on jingle calls. one could presumably alert on DMs, MUC messages, or anything more particular by finding the right Prosody hooks (mod_cloud_notify may be an ok guide for that).

SXMO Integration

SXMO is a mobile-friendly Linux desktop notable for being extremely hackable (it's really just a collection of shell scripts layered over sway), and as such we can integrate all the client-side work we did above into something the desktop environment handles for us transparently.

SXMO includes an autosuspend service. it's just a bash while loop that periodically checks if conditions are suitable to sleep the phone (is the screen off, is the user not in a call, are no media players active, and so on) and if so, calls out to sxmo_suspend.sh.

sxmo_suspend.sh looks like this (simplified -- it does a little more to make cronjobs reliable)

suspend_time=99999999 # far away
sxmo_log "calling suspend with suspend_time <$suspend_time>"

doas rtcwake -m mem -s "$suspend_time" || exit 1

sxmo_hook_postwake.sh

since the different components of SXMO communicate by shelling out to each other, we can patch it just by putting our own sxmo_suspend.sh script somewhere on the PATH. that's intentional behavior: SXMO goes out of its way to add ~/.config/sxmo/hooks as a PATH entry to its services, so we can just drop a file like this into ~/.config/sxmo/hooks/sxmo_suspend.sh:

ntfy sub TEST_WOWLAN_TOPIC &
NTFY_PID=$!

port=
while [ -z "$port" ]; do
  # netstat output will look like:
  #   tcp 0 0 127.0.0.1:12345 10.11.12.13:443 ESTABLISHED 2798004/ntfy
  # pipe through sed/cut to extract the `12345` local port component
  # do this in a loop to allow time for ntfy to establish the connection
  port="$(netstat --tcp --program --numeric-ports | grep ntfy | sed 's/  */ /g' | cut -d' ' -f 4 | cut -d':' -f2)"
done

# configure to wake on any traffic to that ntfy connection
rtl8723cs-wowlan enable-clean
rtl8723cs-wowlan tcp --dest-port "$port"

sxmo_log "calling suspend with suspend_time 600"
doas rtcwake -m mem -s 600 || exit 1
kill $NTFY_PID
sxmo_hook_postwake.sh

wait  # for ntfy to exit

and that's it. use your phone as you normally would, and everything should be the same except that now it'll wake up whenever you get a Matrix/XMPP/ntfy notification.

Pinephone Wowlan Race Condition

as alluded, sending a wake packet within about a second of sending the SoC to sleep will fail to wake the system. worse, the SoC will be stuck in the sleep state, not waking on any future wake packets (you can still wake it via the power button, and it should behave correctly during the next sleep cycle). one can imagine lots of ways an edge-triggered interrupt could be misimplemented to cause this type of race, but i wasn't able to diagnose it any more than this.

in light of that type of thing, i'm being cautious and never sleeping the phone for more than 10 minutes at a time (rtcwake -s 600). so if you get a VoIP call, there's something like a 0.2% chance you'll miss it and see a notification for it 10 minutes later. there's a higher chance than that i miss a call just by forgetting my phone in the other room, so i'm not especially concerned. but if it bothers you, a workaround exists by coordinating your notification server and your phone such that it knows when your phone is about to enter sleep and delays notifications during that time by a couple seconds to avoid triggering the race. here's how i do that.

there are other reasons you may want to not sleep for extended durations: NATs. in theory, a TCP connection with no traffic should remain routable for at least 2 hours -- meaning you could safely sleep for 2 hours and the notification server will still be able to reach you at any point. in practice, i don't know how that holds across all networks. UDP has far lower guarantees (30s to 120s based on who you ask): consider that if you're using a UDP-based VPN like Wireguard. there's also lots of gotchas around WiFi connection details (moving from one network to another, GTK/encryption rekeying, DHCP lease renewals, etc).

in any case, longer sleep durations give diminishing returns. if you sleep for 600s, and then wake for 15s before going back to sleep, you've already captured 95% of the gains you could get from this sleep method. sleeping for an hour isn't going to get you that much more battery life.

Wrapping Up

i only covered the WiFi usecase. the modem also has a line to the CPU and could be made to wake it under similar scenarios. i believe it already does for calls/SMS, but i haven't gotten around to understanding or tuning its behavior for IP traffic (information from anyone who has looked into this would be appreciated!)

i'm not convinced this push notification system will have a place in the long-term future of mobile Linux. in my setup, i'm not actually surfacing any of the content from the push gateway onto the phone's display. the push gateway could just as well be sending the phone 0-byte packets. i'm just using it to wake the phone and then let the applications learn about new messages/calls and tell the DE about them in their ordinary manner. there are simpler ways to do that.

maybe some XDG spec or Wayland protocol will be developed to let applications tell the OS "incoming data on this TCP connection isn't urgent. but data on this other TCP connection is urgent" and then the OS can program the wowlan rules accordingly, and notification servers have no special place. on the other hand, a lot of messaging systems are having to implement Apple/Google's notification system for their users anyway, so am i describing an actual reduction in complexity there or would it more likely be yet another standard everyone would need to support?

maybe you want to help determine that future? if so: Librem's working on their Chatty client, and while i have no special affiliation with them, i'll point out that they're actively considering how to integrate sleep/push notifications into their client recently.

besides that, KDE is also looking into similar things.

Where to Learn More

as always, feel free to message me via any method on my about page.