get_log_sources() named whichever process held a log file with the lowest pid as its writer. A log reader -- CrowdSec's acquisition, Promtail, Filebeat, rsyslog imfile, a tail -F -- holds the file too, so on paperless-01 the inventory said crowdsec writes /mnt/docroot/data/log/paperless.log (NetOrk/netork#2092).
Now only a process holding the file open for writing is its producer:
LOG_SOURCES_COMMAND prints a new [flags] section: the flags: line of /proc/<pid>/fdinfo/<fd> for every descriptor find listed. It is one xargs grep -H over all of them, in the same round trip -- no command per descriptor.
The parse takes the access mode (flags & 3): O_WRONLY/O_RDWR is a writer, O_RDONLY (0100000) a reader. The first writer by pid gives unit and process, whatever pid a reader has.
A file only readers hold stays listed, with unit and process None: it is still a log on the host (the CrowdSec idle check relies on the inventory listing what the agent reads).
A file only a reader inside a container holds is left out. Its path is that container's view of a mount; listing it with container_id kept would give it to the reader's container (the same misattribution), and listing it without would present a container path as a host path. The real file appears in its writer's namespace, or on the host.
A descriptor whose flags did not answer (the process ended in between) is no writer. A report without a [flags] section at all -- an older command -- is read as before, so a recorded fixture still parses the same.
Targets that are no path (socket:[…], pipe:[…]) are never a source.
No interface change; the drivers carry the command unchanged (_run_log_sources_command), so no driver needs a change. The linux, proxmox, openmediavault and qnap-qts driver tests pass against this branch.
Version
Proposed: 4.31.1 (PATCH -- a fix, no interface change). The entry is under [Unreleased].
Tests
Fixtures only, plus the command run on the test host itself:
a writer with 0102001 (O_WRONLY|O_APPEND), a reader with 0100000 and a lower pid on the same file: the writer is named;
a file only CrowdSec reads: listed, no producer, size and last write kept;
an O_RDWR writer (0100002); a reader only inside a container: left out;
a socket and a pipe descriptor: no source; a deleted file still left out though written;
a writer whose fdinfo did not answer; a report without [flags];
live: this test's process holds one file open for appending and one for reading; the command names a producer for the first and none for the second.
## What
`get_log_sources()` named whichever process held a log file with the lowest pid as its writer. A log *reader* -- CrowdSec's acquisition, Promtail, Filebeat, rsyslog `imfile`, a `tail -F` -- holds the file too, so on paperless-01 the inventory said `crowdsec` writes `/mnt/docroot/data/log/paperless.log` (NetOrk/netork#2092).
Now only a process holding the file open **for writing** is its producer:
- `LOG_SOURCES_COMMAND` prints a new `[flags]` section: the `flags:` line of `/proc/<pid>/fdinfo/<fd>` for every descriptor `find` listed. It is one `xargs grep -H` over all of them, in the same round trip -- no command per descriptor.
- The parse takes the access mode (`flags & 3`): `O_WRONLY`/`O_RDWR` is a writer, `O_RDONLY` (`0100000`) a reader. The first writer by pid gives `unit` and `process`, whatever pid a reader has.
- A file only readers hold stays listed, with `unit` and `process` None: it is still a log on the host (the CrowdSec idle check relies on the inventory listing what the agent reads).
- A file only a reader **inside a container** holds is left out. Its path is that container's view of a mount; listing it with `container_id` kept would give it to the reader's container (the same misattribution), and listing it without would present a container path as a host path. The real file appears in its writer's namespace, or on the host.
- A descriptor whose flags did not answer (the process ended in between) is no writer. A report without a `[flags]` section at all -- an older command -- is read as before, so a recorded fixture still parses the same.
- Targets that are no path (`socket:[…]`, `pipe:[…]`) are never a source.
No interface change; the drivers carry the command unchanged (`_run_log_sources_command`), so no driver needs a change. The linux, proxmox, openmediavault and qnap-qts driver tests pass against this branch.
## Version
Proposed: **4.31.1** (PATCH -- a fix, no interface change). The entry is under `[Unreleased]`.
## Tests
Fixtures only, plus the command run on the test host itself:
- a writer with `0102001` (O_WRONLY|O_APPEND), a reader with `0100000` and a lower pid on the same file: the writer is named;
- a file only CrowdSec reads: listed, no producer, size and last write kept;
- an `O_RDWR` writer (`0100002`); a reader only inside a container: left out;
- a socket and a pipe descriptor: no source; a deleted file still left out though written;
- a writer whose fdinfo did not answer; a report without `[flags]`;
- live: this test's process holds one file open for appending and one for reading; the command names a producer for the first and none for the second.
`ruff`, `mypy`, coverage (log_sources 100 %), `napalm-standard check` and `ratchet` pass.
Refs NetOrk/netork#2092
A process that holds a log open only to read it -- CrowdSec's
acquisition, Promtail, rsyslog imfile, a tail -F -- was named as the
file's writer whenever it had the lower pid. The command now reads each
descriptor's flags from /proc/<pid>/fdinfo in the same round trip, and
only an O_WRONLY or O_RDWR holder is the producer. A file only readers
hold stays listed without one; a file only a reader inside a container
holds is its view of a mount and is left out.
Refs: NetOrk/netork#2092
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
What
get_log_sources()named whichever process held a log file with the lowest pid as its writer. A log reader -- CrowdSec's acquisition, Promtail, Filebeat, rsyslogimfile, atail -F-- holds the file too, so on paperless-01 the inventory saidcrowdsecwrites/mnt/docroot/data/log/paperless.log(NetOrk/netork#2092).Now only a process holding the file open for writing is its producer:
LOG_SOURCES_COMMANDprints a new[flags]section: theflags:line of/proc/<pid>/fdinfo/<fd>for every descriptorfindlisted. It is onexargs grep -Hover all of them, in the same round trip -- no command per descriptor.flags & 3):O_WRONLY/O_RDWRis a writer,O_RDONLY(0100000) a reader. The first writer by pid givesunitandprocess, whatever pid a reader has.unitandprocessNone: it is still a log on the host (the CrowdSec idle check relies on the inventory listing what the agent reads).container_idkept would give it to the reader's container (the same misattribution), and listing it without would present a container path as a host path. The real file appears in its writer's namespace, or on the host.[flags]section at all -- an older command -- is read as before, so a recorded fixture still parses the same.socket:[…],pipe:[…]) are never a source.No interface change; the drivers carry the command unchanged (
_run_log_sources_command), so no driver needs a change. The linux, proxmox, openmediavault and qnap-qts driver tests pass against this branch.Version
Proposed: 4.31.1 (PATCH -- a fix, no interface change). The entry is under
[Unreleased].Tests
Fixtures only, plus the command run on the test host itself:
0102001(O_WRONLY|O_APPEND), a reader with0100000and a lower pid on the same file: the writer is named;O_RDWRwriter (0100002); a reader only inside a container: left out;[flags];ruff,mypy, coverage (log_sources 100 %),napalm-standard checkandratchetpass.Refs NetOrk/netork#2092