Hi guys,
my worker daemon is trying to do some job that is making my system go OOM literally every 5 minutes.
Problem is, that worker runs, then goes nuts and consume 4 cores and all available memory (+-2,5Gigs), then OOMs. Nothing i tried had any effect.
I run 2025.07rc from YuNoHost, but had same issue on 2024.12
I would like to ask you for guidance - I guess I need to pin-point the job and propably delete it maually (or do it from my desktop with 32Gigs of RAM remotely).
Things I tried so far
Upgrading to 2025
Reduce worker threads
Reduce queries per worker job
Reduce buffer and packet sizes for MariaDB and PHP
Reduce pm.max_children
Enabling database cleanup in admin panel
Friendica Developers reshared this.
Michael 🇺🇦
in reply to Schmaker • • •Schmaker
in reply to Michael 🇺🇦 • •Have no idea where to get the callstack, so I tried my best bet 😀
Feel free to point me in the right direction, I'm away from my terminal so I tried to get these from web access at least...
dmesg
/var/log/syslog
Friendica Developers reshared this.
Michael 🇺🇦
in reply to Schmaker • • •Schmaker likes this.
Schmaker
in reply to Michael 🇺🇦 • •I'm not that sure as the process does not respect memory limit (you can safely ignore this being php8.4 instead of 8.2 earlier - i just fixed my conf to make sure it's the same).
Edit: Figured out why it consumes all memory - php-cli has it's own ini file.
Friendica Developers reshared this.
Michael 🇺🇦
in reply to Schmaker • • •Schmaker likes this.
Schmaker
in reply to Michael 🇺🇦 • •As edited - I just found out I need to set limit for php-cli as well. My bad, sorry about that.
Now, where should I look for the callstack?
Friendica Developers reshared this.
Michael 🇺🇦
in reply to Schmaker • • •Schmaker
in reply to Michael 🇺🇦 • •Not sure what to look for, only thing I do see there is:
2025-12-03T19:08:11Z worker [ALERT]: Fatal Error (E_ERROR): Allowed memory size of 1073741824 bytes exhausted (tried to allocate 16384 bytes) {"code":1,"message":"Allowed memory size of 1073741824 bytes exhausted (tried to allocate 16384 bytes)","file":"/var/www/friendica/src/Database/Database.php","line":543,"trace":null,"worker_id":"d142b9f","worker_cmd":null} - {"file":null,"line":null,"function":null,"request-id":"69308a8f0f968","stack":"","uid":"5624c8","process_id":8088}Limit for php-cli is 1024M
pastebin.com/D64dbJhn
Friendica Developers reshared this.
Michael 🇺🇦
in reply to Schmaker • • •d142b9f. They belong together.Schmaker
in reply to Michael 🇺🇦 • •Friendica Developers reshared this.
Michael 🇺🇦
in reply to Schmaker • • •worker_idin the log line with the exception. Then collect all entries with the same value.Schmaker
in reply to Michael 🇺🇦 • •I just checked in a hurry and it does not seems like there are any new worker related
DEBUGlines in the log. I'll let it run and check back in the evening after work.Thank you for your asistance.
Friendica Developers reshared this.
Michael 🇺🇦
in reply to Schmaker • • •Schmaker
in reply to Michael 🇺🇦 • •@Michael 🇺🇦
That's what I did, but in the morning - even though memory limit was reached - there was no related debug line.
Hoping for different outcome in the evening 😀
@Friendica Developers
Friendica Developers reshared this.
Schmaker
in reply to Michael 🇺🇦 • •next.oscloud.cz/s/Ft4mJaCEZwEP…
Friendica Developers reshared this.