93 lines
5.7 KiB
Markdown
93 lines
5.7 KiB
Markdown
### Progress
|
||
- [x] Zeroth things first, let's create a simple CLI application with `--verbosity VAL` and `--help` options.
|
||
- [x] First things first, let's implement something that reacts when some provided file changes (not poll please).
|
||
- [x] Second things second, implement a simple logging system with differing levels of verbosity and configurable
|
||
output file using cli options.
|
||
- [x] Third things third, implement a thing that simultaneously watches two different files (multithreading).
|
||
it should be cancellable with ctrl+c, but it should just contiuously print event notifications.
|
||
- [x] Fourth things fourth, implement a prototype that reads a space-separated file and populates a struct.
|
||
- [x] Fifth things fifth, implement a prototype that spawns a new thread that executes a shell command.
|
||
- [x] Sixth things sixth, daemonize it!
|
||
- [ ] Seventh things seventh, package the sucker (arch, debian, alpine, docker)
|
||
- [x] archlinux
|
||
- https://wiki.archlinux.org/title/Creating_packages
|
||
- [x] debian
|
||
- see `/home/agj/documents/Projects/packaging/deb-packaging-tutorial.pdf`
|
||
- just use docker.
|
||
- [-] ~~alpine~~ later.
|
||
- [-] ~~docker~~ later.
|
||
- [ ] Eight things eight, try it out! - maybe even write the python webhook extension.
|
||
- [ ] Ninth things ninth, fix bugs, see below
|
||
- [ ] Tenth things tenth, write manpages, choose license
|
||
- [ ] Eleventh things Eleventh, polish
|
||
- [ ] Twelveth things last, release!
|
||
- Setup gitea.gtz.dk (will learn you how to set up subdomains (useful for shop.gtz.dk))
|
||
|
||
BOOKMARK: okay. Now it feels like it's getting complicated. I want to run `sci` in a docker container. But that means
|
||
that the build-threads also run in that docker container - meaning the container should have all the build dependencies
|
||
installed and we all know where that rabbithole goes. 9-30YB docker images with about a trillion unique build systems.
|
||
Let's not do that.
|
||
The only alternative I can see is that the `sci` service is just not dockerized. The pipeline scripts can easily be
|
||
dockerized themselves. Just have a `scripts/wget-src-dist-and-sci-sh-dockerized.sh` with `arg1` being the docker image
|
||
to use?
|
||
```sh
|
||
#!/bin/sh
|
||
wget "$SCI_PIPELINE_URL"
|
||
docker run --rm -it --mount type=bind,source="$(pwd)"/thefileyouwgot.tar.gz,target=/thefileyouwgot.tar.gz,readonly --entrypoint sh $2
|
||
```
|
||
Or something like that... Perhaps we can figure something out with an inline `ADD`, that also extracts the archive in
|
||
the container or something. This approach is cleaner IMO. You can also more easily edit the `pipelines.conf` file if you
|
||
need to.
|
||
|
||
You were getting the following `pipelines.conf` file to work:
|
||
```
|
||
scih-dev ssh://git@git.gtz.dk:222/agj/scih.git scih-onpush /etc/sci/scripts/git-clone-and-run-sci-sh.sh
|
||
scih-release ssh://git@git.gtz.dk:222/agj/scih.git scih-onrelease /etc/sci/scripts/git-clone-and-run-sci-sh.sh
|
||
```
|
||
To get this to work, you need to change some things regarding the packaging - i.e. the `scripts` directory should be
|
||
installed as well. Also, you want `sci` to be containerizable.
|
||
For the `.sci.sh` script to `docker login`, it will need secrets. Implement those.
|
||
|
||
You were reading :Man system.unit and :Man systemd.service as preperation on making a systemd unit file
|
||
This will be needed for the .deb package, as well as the arch linux package.
|
||
alpine linux is using OpenRC (cool), which complicates things a little bit, but shouldn't be too bad. The wiki is
|
||
generally really well written. Otherwise, I am sure that both wiki.gentoo and wiki.archlinux have great pages too
|
||
docker is super easy, just make a dockerfile - only concern is the trigger files.
|
||
|
||
#### Bugs / Missing Features
|
||
- [x] command output is being inherited. It should be piped into some random log-file
|
||
- [ ] pretty sure that `ctrl+c` / SIGINT is not graceful yet.
|
||
- [ ] missing license (heavily considering GPLv3)
|
||
- [ ] pipeline scripts should be executed in a unique `/tmp` dir
|
||
- [ ] Some way for third parties to see which pipelines are currently running and their status.
|
||
- Could be as simple as looking in the logs directory.
|
||
- How to mark a run as failed / success / warn?
|
||
- Third parties may need to extract artifacts.
|
||
or maybe the scripts themselves would upload artifacts?
|
||
- [ ] I am deliberately not using `Restart=on-failure` in the `scid.service` file because we are using `Type=exec`
|
||
and not `Type=notify` (yet) - which would require a `sd_notify` call of `READY=1` (see `man systemd.service`)
|
||
- [ ] Custom environment variables passed to the pipelines on invokation should be possible.
|
||
- [ ] Listener threads should be killed and restarted (worker pool should just chug along) when pipeline config file
|
||
has changed during runtime. Should be disableable with `--no-hot-reload-config` - i.e. on by default.
|
||
- [ ] `docker stop` is very slow. I am probably not handling signals properly yet.
|
||
- [x] It seems that `-v 4` is segfaulting when running release builds, maybe the logger just cant find the source file?
|
||
Nope. I just wrote some bad code (inverted NULL check).
|
||
|
||
### Note Regarding `inotify` usage
|
||
From the manpage:
|
||
```
|
||
With careful programming, an application can use inotify to efficiently monitor and cache the state of a set of
|
||
filesystem objects. However, robust applications should allow for the fact that bugs in the monitoring logic or races
|
||
of the kind described below may leave the cache inconsistent with the filesystem state. It is probably wise to do some
|
||
consistency checking, and re‐build the cache when inconsistencies are detected.
|
||
```
|
||
i.e., we should _also_ poll the watched files every once in a while (maybe once per minute? idk) to ensure that we catch
|
||
all events.
|
||
|
||
### Useful manpages
|
||
- `hier(7)`
|
||
- `daemon(7)` and `daemon(3)`
|
||
- `PKGBUILD(5)`
|
||
- `info make`
|
||
- `https://wiki.archlinux.org/title/Creating_packages`
|