agj/blog
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: descope the portainer post

Browse Source 
crowdsec is a separate adventure.
This commit is contained in:
Asger Gitz-Johansen 2025-04-15 21:38:35 +02:00
parent c25a925a5e
commit 5e11172576
2 changed files with 326 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

210
content/posts/how-to-crowdsec.md Normal file
View File

@ -0,0 +1,210 @@
+++
date = '2025-04-15'
draft = true
title = "How to Set Up Crowdsec"
tags = ["howto", "tutorial", "web", "securoty"]
categories = ["technical"]
+++
## Crowdsec
> NOTE: This configuration blocks *many* varieties of clients and services. You might want to whitelist your own ISP and
> / or your own IP ranges (perhaps even your entire country if you're trusting enough) in case your own services and
> homebrew experiments gets banned.
Short for [crowdsecurity](https://www.crowdsec.net/), crowdsec is a community effort to bring auto-banning security to
the masses, and it's surprisingly easy to set up. You just have to understand how the thing works.
I noticed that I am getting a lot of suspicious traffic on my gitea instance. Usernames such as `log4j` and `thomad`
from china and bulgaria. Yea. Let's enable some fucking security.
Fuck I hate that I have to do this, but I guess people will be assholes.
After [this](https://www.youtube.com/watch?v=-GxUP6bNxF0), the banhammer came down with the might of zeus. Now no-one
gets access. Not even me. I tried to do some country-code whitelisting, but that was a bit of a dud. I'm tired now.
Will look at it tomorrow.
Okay! I seem to have it working now! That was an adventure. Will elaborate when I get back home.
Okay, There's multiple "things" to a crowdsec setup. Crowdsec (the non-paid cloud solution) consists of:
- The core crowdsec security engine (`crowdsecurity/crowdsec` container image)
- Does the "detection" and hardcore logic and makes decisions.
- The bouncer (`fbonalair/traefik-crowdsec-bouncer:latest` container image in my case)
- Enforces the decisions.
- There are multiple different "types" of bouncers. I just use the forwardAuth type, as that is the most straight
forward one. Especially when combined with traefik.
### Concepts
In short, there are only a couple of concepts you should know in order to *use* crowdsec. This is
Feel free to skip these if you
don't care for now, and just want something up and running.
- Acquisitions
- In order for `crowdsec` to know *what* and *where* to look for potential intruders, threats etc. You must tell it
in the form of *acquisition* configurations. The easiest thing to do is to just give `crowdsec` access to your docker
logs and traefik logs - this is excactly what we're aiming to do.
- Parsers
- Bucket Overflow
- Bouncers
Note that the core crowdsec security engine should be part of the core traefik/portainer deployment because it will
need some elevated privileges. The traefik service should also register some middlewares, so it can't be part of the
portainer managed containers / stacks.
When using Traefik, make sure to add the docker labels that enable traefik trafficing to the containers:
```yaml
# For the new containers.
labels:
- traefik.enable=true
- traefik.docker.network=proxy
- traefik.http.routers.traefik-bouncer.entrypoints=websecure
```
## "easy"
This shit was not easy to set up. But it is easy to maintain. Keep a "new"/"learning" mind, and all should be fine.
## Configuring the Bouncer
Also called "Remediation"
I am using the traefik bouncer, that is using
[forwardAuth](https://doc.traefik.io/traefik/middlewares/http/forwardauth/) to check if an IP is blocked or not.
Configure the container in docker compose and afterwards, you should introduce the traefik middleware in the dynamic
and static configuration, like so:
```yaml
# dynamic traefik config
http:
middlewares:
traefikBouncer:
forwardauth:
address: http://traefik-bouncer:8080/api/v1/forwardAuth
trustForwardHeader: true
```
```yaml
# static traefik config
entryPoints:
http:
address: ":80"
http:
middlewares:
- traefikBouncer@file
https:
address: ":443"
http:
middlewares:
- traefikBouncer@file
```
If you have (I do) some other names for the `address: ":443"` and `":80"` middlewares, don worry, just add the
`traefikBouncer@file` to the list of middlewares and you should be good.
You will have to register your bouncer through the `cscli` as well:
```sh
docker exec crowdsec cscli bouncers list
docker exec crowdsec cscli bouncers add traefikBouncer
```
This should give you an API key. Place it in the environment variable `CROWDSEC_BOUNCER_API_KEY: <your-key-here>`.
Additionally, you should add the `CROWDSEC_AGENT_HOST: crowdsec:8080` environment variable (assuming the crowdsec
docker _service_ is called `crowdsec`) - the port is standard and you don't need to portmap or expose anything btw.
### Crowdsec Core Security Engine Configuration
In order for the crowdsec security engine to be able to detect intruders, it needs access to the logs of the other
containers on the server. To do this, you can just volume mount: `/var/run/docker.sock:/var/run/docker.sock:ro` and
then
Check out [https://app.crowdsec.net/hub/configurations](https://app.crowdsec.net/hub/configurations) if there are logparsers available for the service you want
to integrate.
#### Acquisitions
In the `acquis.d` directory (volume mapped into the `crowdsec` container to `./acquis.d:/etc/crowdsec/acquis.d`),
you should add YAML files for each source you want the crowdsec engine to scan for criminals and other scum:
```txt
acquis.d/
├── gitea.yaml
└── traefik.yaml
```
File Contents:
```yaml
# traefik.yaml
filenames:
- /var/log/traefik/*
labels:
type: traefik
```
```yaml
# gitea.yaml
source: docker
container_name:
- gitea
labels:
type: gitea
```
`traefik.yml` is a `filename` based acquisition file, meaning you need to configure the `traefik` container to
output access and system logs into a directory that is volume-mapped so that it's available to the crowdsec
container (`traefik-logs:/var/log/traefik/:ro` and associated `traefik-logs:/var/log/traefik/` on traefik).
The acquisition file for the `gitea` service is using the `docker` source. So it'll read the `docker logs`. The
cool thing about this, is that you dont have to do any extra configuration on the gitea side.
To configure `traefik` to output logs into a file (default it just outputs to stdout/stderr for no-one to read),
add the following to your static config (`traefik.yml`) - make sure to `docker compose up -d --force-recreate`
every time you edit the config (and want to apply the changes):
```yaml
# ... at the end of traefik.yml
log:
level: INFO
filePath: /var/log/traefik/traefik.log
accessLog:
filePath: /var/log/traefik/access.log
```
Also, in docker compose file, install some collections:
```yaml
# in crowdsec container spec
environment:
GID: "$(GID-1000)"
COLLECTIONS: "crowdsecurity/linux crowdsecurity/traefik crowdsecurity/whitelist-good-actors LePresidente/gitea"
```
#### Geofenching
You might have lost the bouncer - check with `docker exec crowdsec cscli bounders list`.
I am hosting some services that may produce some false-flags by crowdsec, so I will be whitelisting my country. To
do this, we need to register a country-code whitelist
[postoverflow](https://docs.crowdsec.net/docs/whitelist/create_postoverflow/) in the `postoverflows` directory,
which is volume mapped `./postoverflows:/etc/crowdsec/postoverflows/`:
```yaml
# postoverflow/s01-whitelist/sc-countries-whitelist.yaml
name: my/whitelist
description: Whitelist trusted regions
whitelist:
reason: Whitelisted country
expression:
- "evt.Enriched.IsoCode == 'DK'" # NO! Not anymore!
```
Note that the data is not "enriched" with the IsoCode yet. You need to install the `geoip-enrich` thing:
```sh
docker exec crowdsec cscli parsers install crowdsecurity/geoip-enrich
```
This solution is not very sophisticated, so I might change this to something less "sledgehammer"-y in the future.

125
content/posts/how-to-portainer.md
View File

@ -1,26 +1,133 @@
+++
date = '2024-12-04'
date = '2025-04-14'
draft = true
title = "How to Host Docker Containers Easily in The Cloud"
tags = ["howto", "tutorial", "web"]
categories = ["technical"]
+++
In this post, we will be going over how to set up a [portainer]() managed docker environment, and how to use it.
This is ideal if you want to host a personal website, a [blog](/posts/how-to-blog), a personal [github](git.gtz.dk) or whatever your development heart desire.
If you choose to follow along, by the end of it, you will have an environment where you can just add or remove docker based services. It's even quite secure!
In this post, we will be going over how to set up a [portainer](https://www.portainer.io/) managed docker environment,
and how to use it. This is ideal if you want to host a personal website, a [blog](/posts/how-to-blog), a personal
[github](git.gtz.dk) or whatever your development heart desire.
If you choose to follow along, by the end of it, you will have an environment where you can just add or remove docker
based services at a whim.
## Portainer
I assume that you already know about `docker` and `docker compose` yaml syntax. If you don't, may I recommend the
wonderful official [docker tutorial](https://docs.docker.com/get-started/workshop/) - once you're done with that come
back here. Or just read on and roll with the punches.
## Traefik
Oh yea, you should also have good knowledge and experience working on GNU/Linux systems, as you'll be doing a lot of
management and interaction with the terminal both during the setup process and during maintenance.
## Server
The very first thing to get is a server. This can either be the machine you're currently using if you don't want to mess
around on the public internet, or it could be an actual desktop you have set up with a public IP. Or it could be a VPS
(Virtual Private Server) - which is just a fancy word for a "cloud computer" that someone else hosts and powers, and you
just get an SSH connection to it. Any VPS provider will work, but [digital ocean](https://www.digitalocean.com/) is very
affordable and easy to use. As long as you get a VPS and avoid a *webhotel*, you should be fine (side note: webhotels
are a scam and you shouldn't ever use them - especially not if you're tech-savvy enough to read this blog).
Once you have your server, [install](https://docs.docker.com/engine/install/) docker on it. Preferably the latest
version.
## Traefik and Portainer
The very first thing to get done is set up portainer and traefik. This is done by creating a new `docker-compose.yml`
file on your server. Just to keep things tidy, you should make a directory for all you are going to do here.
```sh
# Make the config directory in your $HOME dir - this is where
# we'll be working throughout the tutorial. If not specified
# otherwise, you should only be editing files inside this directory.
mkdir -p ~/config
mkdir -p ~/config/traefik-data
mkdir -p ~/config/portainer-data
cd ~/config
# Create an empty yaml file
touch docker-compose.yml
```
It might be a good idea to initialize the `control` directory as a (local) `git` project. That way you will always have
a history of what you have been done, and what you did when you (inevitably) break things. This I will leave up to you
though (probably gitignore the `portainer-data` directory).
Inside the new `docker-compose.yml` file, you should put the following content (open the file using your favorite
terminal text editor).
```yaml
# docker-compose.yml
services:
traefik:
image: traefik:latest
container_name: traefik
restart: unless-stopped
security_opt:
- no-new-privileges:true
networks:
- proxy
ports:
- 80:80
- 443:443
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik-data/traefik.yml:/traefik.yml:ro
- ./traefik-data/acme.json:/acme.json
- ./traefik-data/configurations:/configurations
labels:
- traefik.enable=true
- traefik.docker.network=proxy
- traefik.http.routers.traefik-secure.entrypoints=websecure
- traefik.http.routers.traefik-secure.rule=Host(`traefik.example.com`)
- traefik.http.routers.traefik-secure.service=traefik
- traefik.http.routers.traefik-secure.middlewares=user-auth@file
- traefik.http.routers.traefik-secure.service=api@internal
portainer:
image: portainer/portainer-ce:alpine
container_name: portainer
restart: unless-stopped
security_opt:
- no-new-privileges:true
networks:
- proxy
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./portainer-data:/data
labels:
- traefik.enable=true
- traefik.docker.network=proxy
- traefik.http.routers.portainer-secure.entrypoints=websecure
- traefik.http.routers.portainer-secure.rule=Host(`portainer.example.com`)
- traefik.http.routers.portainer-secure.service=portainer
- traefik.http.services.portainer.loadbalancer.server.port=9000
networks:
proxy:
external: true
```
Whew! That's a lot. Let's break it down. We define two services `traefik` and `portainer`. Starting with the things that
are common to both of them, we set the initial niceties, such as the `container_name`, restart policy, security options
and set their shared network to be the externally defined `proxy` network. Both services need (read-only) access to the
system time for various reasons, so we volume mount `/etc/localtime` to their respective internal `/etc/localtime`. They
also both need access to the system docker socket, so we also volume mount that in (again, read-only). Then we map the
various configuration files in (we will soon make these).
If you haven't used `traefik` before, you might be scratching your head on the `labels` that we set on each of the
services. This is how you configure services to integrate into traefik, enabling you to route your various containers to
various subdomains, integrate middlewares such as forcing HTTPS and setting load-balancer settings etc.
Let's add the configuration files, shall we?
## Keycloak
## Automatic backups
## TODOs
- [ ] 2FA the control dashboards through keycloak
- [ ] geoblocking the control dashboards
- [x] geoblocking the control dashboards
- [ ] start the article with a demo of what we'll be making
- MAYBE:
- [ ] portainer introduction (maybe)