feat: I found my old blog

It only had 2 posts, but whatever
This commit is contained in:
Asger Gitz-Johansen 2025-04-13 19:48:15 +02:00
parent 796f5ac759
commit 479a1a3d27
8 changed files with 1880 additions and 0 deletions

3
Makefile Normal file
View File

@ -0,0 +1,3 @@
demo:
docker build -t wip .
docker run --rm -it -p 8080:8080 wip hugo serve --bind 0.0.0.0 --port 8080

View File

@ -19,3 +19,7 @@ docker run --rm -it -p 8080:8080 wip hugo serve --bind 0.0.0.0 --port 8080
- [ ] how to securely "self-host" using a VPS, portainer and traefik
- [ ] how to configure neomutt
- [ ] how to securely host a mail server
### Old sillyblog
- [x] Avr memory model
- [x] similarity graph

View File

@ -0,0 +1,110 @@
+++
date = '2019-03-24'
draft = false
title = 'AVR Memory Model: The Practical Explanation'
tags = ['avr', 'programming', 'memory']
categories = ['technical']
+++
There is A LOT of people that have already explained this, but I personally don't feel like going through billions of
forum posts, with most of them just dying out somewhere in 2008 with no better answer than "I figured it out, thanks".
Regardless. BIG shoutout to the amazing people at [AVR Freaks](https://www.avrfreaks.net/). They are really cool.
Seriously. Make a user and ask them about anything, and they'll help you.
## Disclaimer
I have only been debugging the memory usage of a specific ATMega chip. I don't know if other AVR chip-types use the
same, but this explanation should be valid for all MCUs that the
[avr-libc](http://www.nongnu.org/avr-libc/user-manual/index.html) package supports.
I also assume that GNU/Linux is being used on the development computer.
### Open-Source development tools
The [avr-gcc](http://www.nongnu.org/avr-libc/user-manual/pages.html) compiler chain is an open source effort to have
C/C++ for AVR Atmel chips. They do provide some rudimentary C++ support, but there's no STL and the `new` and `delete`
keywords are not implemented by default. Even purely virtual functions doesn't work out of the box.
**But don't fret!** There are ways to implement those features manually. See my [other]() post about getting the build
environment up and running.
## The Memory Model
As the avr-libc developers [explain](http://www.nongnu.org/avr-libc/user-manual/malloc.html), there's typically not a
lot of RAM available on most many devices and therefore it's very important to keep track of how much memory you are
using.
{{< centered image="/malloc-std.png" >}}
All of these symbols `SP`, `RAMEND`, `__data_start`, `__malloc_heap_start`, etc. Can be modified in the compiler, but
the picture above gives the default layout (for an ATMega128 MCU). It goes without saying, that if you don't have an
external RAM chip, you won't be able to utilize the extra RAM space for that. Otherwise, the memory addresses are pretty
straight forward: `0x0100 => 256` bytes is the start of the memory, `0x10FF => 4351` bytes is the end. If you're
wondering where the RAM ends on your specific MCU, you can usually simply open the spec-sheet of the chip and see the
amount of available memory is in it.
For the [ATMega128](https://www.microchip.com/wwwproducts/en/ATMEGA128) that number is 4096 (`4351 - 256 = 4095` (the
spec-sheet also counts the 0th byte)).
## The avr-libc Memory Allocators
Now for the juicy part. whenever you `malloc` something in your program, the allocator first writes a 2-byte *free-list
entry* that tells the system how big your object is.
Example:
```cpp
/* ... */
// Allocate an array of 5 integers
int* my_heap_object = static_cast<int*>(malloc(sizeof(int) * 5));
/* ... */
```
Assuming that the memory has been cleared on chip-startup, the above example ends up with the memory setup looking like
this: (Don't mind the specifc memory addresses. If you're curious, you can try doing this, by attaching `avr-gdb` to a
simulator or On Chip Debugger (OCD)).
```
gdb: > x/16xb my_heap_object
0x800100: 0a 00 00 00 00 00 00 00
0x800108: 00 00 00 00 00 00 00 00
```
The first bytes at address `0x800100` are `0a` and `00`. These bytes are the *free-list* entry and explains how "big"
the object is. When reading this, we have to remember that the model is littleengine-based (meaning that the bytes are
switched),
so we actually have the value of `0x000a`, meaning `10` in decimal. This makes a lot of sense, since we allocated 5
`int`s, that is of size 2 (16bit integers).
The memory dump shows 16 bytes in total, so the last 4 bytes displayed in the gdb example are not part of the object.
However, if you look at the Memory Model picture again, you can see that the `__brkval` value points to the biggest
memory address that has not been allocated. In our example, if you check where the `__brkval` points to after our
allocation, we get:
```
gdb: > __brkval
$ 268
```
268 in hexadecimal is `0x10c`, and if interpreted as an address we get `0x80010c`, which fits very well with our
example, since it is exactly 12 bytes away from where the free-list entry of `my_heap_object` is located at.
When `free`-ing the object again, the deallocator looks at the free-list entry at the given address, and wipes the
free-list entry. **This is why you should not free a dangling pointer**. Freeing something that is not really free-list
entry *will* result in undefined behaviour, and I think we all know how bad **that** is. (Even though the AVR
environment is actually very good at handling it. In my experience, it usually just crashes and starts over.)
However, as
[explained](http://www.nongnu.org/avr-libc/user-manual/group__avr__stdlib.html#gafb8699abb1f51d920a176e695ff3be8a) in
the avrlibc documentation, freeing the `NULL` value, doesn't do anything. So remember to assign your free'd pointers to
`NULL` afterwards.
## Wrapping up
The memory allocators of AVR can be very confusing and if you don't keep your thoughts straight when programming, you
can very easily get yourself into a lot of trouble. Since STL is not available to avr-gcc programmers, we dont have our
glorious smart pointers, so we should implement them ourselves (or use arduino's implementations). That might become a
future blogpost.
Regardless, I hope this helps the lost souls that are trying to actually use these tools.
{{< centered image="/6616144.png" >}}

View File

@ -0,0 +1,103 @@
+++
date = '2019-03-13'
draft = false
title = 'Plotting Circular Bubble Graph in LaTeX'
tags = ['latex', 'programming']
categories = ['technical']
+++
When doing web-intelligence, you might need to visualize what's called inter-sentence similarity in a particular format.
I personally haven't found an official name for these kinds of graphs, so I just simply call them Circular Bubble
Graphs.
During a university project we needed such a graph in our report, and I got the idea of automatically plotting it
through `gnuplot` and integrating it directly into our report with `gnuplottex`. You can see an example of the outcome
of the script.
![Sentence similarity bubble graph example](/sentence_similarity_graph_example.svg)
The script operates on a comma-seperated file (`.csv`). The data should be provided as a matrix of sentences assumed to
be symmetric, with the cells containing a real number from 0 to 1, indicating the similarity between the column and the
row. Because of this symmetric property, half of the matrix is ignored by the script. (It also ignores the diagonal,
since sentence `23` will always have a maximum similarity to sentence `23`. It would also be hard to plot that line)
The whole script can be seen below, but you can also download it as a file
[here](/sentence_similarity_graph.gnuplot). Make sure to set the `my_dataset` variable to your desired
dataset. Example matrix can be downloaded [here](/example_similarities.csv).
```bash
# Sentence similarity graph plotter
# uncomment this for manual operation of the dataset plotted
# my_dataset = "./sentence_similarities.csv" # ARG1
set parametric
set size square
# Styling
set pointsize 7.5
set style fill solid 1.0 border rgb 'grey30'
set style line 1 lc rgb 'black' pt 6 lw 0.5
# Basically a one-dimensional circular coordinate system
fx(t) = cos(t)
fy(t) = sin(t)
rownum = floor(system("wc -l ".my_dataset."")) +1
coord(k) = (k/real(rownum))*(2*pi)
fxx(t) = cos(coord(t))
fyy(t) = sin(coord(t))
set trange [0:2*pi-(coord(1.0))]
set sample rownum
set noborder
unset tics
set xrange [-1.2:1.2]
set yrange [-1.2:1.2]
set title "Sentence inter-similarity graph"
set multiplot
refloptimization = 0
do for [i = 0:rownum-1] {
do for [j = refloptimization:rownum-1] {
if (i != j) {
# Get how many columns there are in the dataset.
arrwidth = real(system("awk 'FNR == ".(i+1)." {print $".(j+1)."}' ".my_dataset.""))
if (arrwidth > 0.0) {
bubblerad = 0.125
x1 = fxx(i)
y1 = fyy(i)
x2 = fxx(j)
y2 = fyy(j)
dvx = x2-x1
dvy = y2-y1
dvl = sqrt((dvx ** 2) + (dvy ** 2))
x1 = x1 + (dvx/dvl)*bubblerad
y1 = y1 + (dvy/dvl)*bubblerad
x2 = x2 - (dvx/dvl)*bubblerad
y2 = y2 - (dvy/dvl)*bubblerad
# Overleaf's arrow-width rendering is pretty terrible,
# so we use a color-gradient to determine connection-strength.
if (arrwidth > 0.2) {
col = "#000000"
} else {
if (arrwidth < 0.1) {
col = "#B8B8B8"
} else {
col = "#E4E4E4"
}
}
set arrow "".i.j."" from x1,y1 to x2,y2 nohead lw 0.5 lc rgb col
#set label "H" at (fxx(j)-fxx(i)),(fyy(j)-fyy(i))
show arrow "".i.j.""
}
}
}
refloptimization = refloptimization + 1
}
# Plot the circles
plot '+' u (fx(t)):(fy(t)) w p ls 1 notitle
# Plot the sentence labels
plot '+' u (fx(t)):(fy(t)):(sprintf("s.%d",$0+1)) with labels notitle
```
{{< centered image="/6616144.png" >}}

21
static/example_similarities.csv Executable file
View File

@ -0,0 +1,21 @@
1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0492 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2353 0.0000 0.0000 0.2182
0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 1.0000 0.0000 0.1694 0.0510 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1186 0.0371 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.1694 0.0000 1.0000 0.2748 0.0286 0.2232 0.2697 0.1424 0.0000 0.0000 0.2443 0.2312 0.0000 0.1602 0.0000 0.0000 0.2273 0.1699
0.0000 0.0000 0.0000 0.0510 0.0000 0.2748 1.0000 0.0480 0.0627 0.0761 0.0432 0.0000 0.0000 0.2907 0.1857 0.0000 0.0543 0.0000 0.0000 0.0737 0.0517
0.0492 0.0000 0.0000 0.0000 0.0000 0.0286 0.0480 1.0000 0.0520 0.0000 0.0634 0.0000 0.0739 0.1664 0.1848 0.1117 0.0370 0.2022 0.0739 0.2066 0.1055
0.0000 0.0000 0.0000 0.0000 0.0000 0.2232 0.0627 0.0520 1.0000 0.0000 0.0000 0.0000 0.0000 0.1581 0.0518 0.0000 0.0000 0.0597 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.0000 0.0000 0.2697 0.0761 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.2148 0.0595 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.0000 0.0000 0.1424 0.0432 0.0634 0.0000 0.0000 1.0000 0.0000 0.0000 0.1091 0.0304 0.0000 0.0000 0.0391 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0739 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.1010 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.1186 0.0000 0.2443 0.2907 0.1664 0.1581 0.2148 0.1091 0.0000 0.0000 1.0000 0.2803 0.0000 0.1291 0.0000 0.0000 0.2894 0.1543
0.0000 0.0000 0.0000 0.0371 0.0000 0.2312 0.1857 0.1848 0.0518 0.0595 0.0304 0.0000 0.0000 0.2803 1.0000 0.0000 0.0375 0.0560 0.0000 0.2090 0.0748
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1117 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0830 0.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.0000 0.0000 0.1602 0.0543 0.0370 0.0000 0.0000 0.0000 0.0000 0.0000 0.1291 0.0375 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000
0.2353 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2022 0.0597 0.0000 0.0391 0.0000 0.1010 0.0000 0.0560 0.0830 0.0000 1.0000 0.0000 0.0000 0.2176
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0739 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000
0.0000 0.0000 0.0000 0.0000 0.0000 0.2273 0.0737 0.2066 0.0000 0.0000 0.0000 0.0000 0.0000 0.2894 0.2090 0.0000 0.0000 0.0000 0.0000 1.0000 0.0650
0.2182 0.0000 0.0000 0.0000 0.0000 0.1699 0.0517 0.1055 0.0000 0.0000 0.0000 0.0000 0.0000 0.1543 0.0748 0.0000 0.0000 0.2176 0.0000 0.0650 1.0000
1 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0492 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2353 0.0000 0.0000 0.2182
2 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
3 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
4 0.0000 0.0000 0.0000 1.0000 0.0000 0.1694 0.0510 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1186 0.0371 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
5 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
6 0.0000 0.0000 0.0000 0.1694 0.0000 1.0000 0.2748 0.0286 0.2232 0.2697 0.1424 0.0000 0.0000 0.2443 0.2312 0.0000 0.1602 0.0000 0.0000 0.2273 0.1699
7 0.0000 0.0000 0.0000 0.0510 0.0000 0.2748 1.0000 0.0480 0.0627 0.0761 0.0432 0.0000 0.0000 0.2907 0.1857 0.0000 0.0543 0.0000 0.0000 0.0737 0.0517
8 0.0492 0.0000 0.0000 0.0000 0.0000 0.0286 0.0480 1.0000 0.0520 0.0000 0.0634 0.0000 0.0739 0.1664 0.1848 0.1117 0.0370 0.2022 0.0739 0.2066 0.1055
9 0.0000 0.0000 0.0000 0.0000 0.0000 0.2232 0.0627 0.0520 1.0000 0.0000 0.0000 0.0000 0.0000 0.1581 0.0518 0.0000 0.0000 0.0597 0.0000 0.0000 0.0000
10 0.0000 0.0000 0.0000 0.0000 0.0000 0.2697 0.0761 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.2148 0.0595 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
11 0.0000 0.0000 0.0000 0.0000 0.0000 0.1424 0.0432 0.0634 0.0000 0.0000 1.0000 0.0000 0.0000 0.1091 0.0304 0.0000 0.0000 0.0391 0.0000 0.0000 0.0000
12 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
13 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0739 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.1010 0.0000 0.0000 0.0000
14 0.0000 0.0000 0.0000 0.1186 0.0000 0.2443 0.2907 0.1664 0.1581 0.2148 0.1091 0.0000 0.0000 1.0000 0.2803 0.0000 0.1291 0.0000 0.0000 0.2894 0.1543
15 0.0000 0.0000 0.0000 0.0371 0.0000 0.2312 0.1857 0.1848 0.0518 0.0595 0.0304 0.0000 0.0000 0.2803 1.0000 0.0000 0.0375 0.0560 0.0000 0.2090 0.0748
16 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1117 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0830 0.0000 0.0000 0.0000
17 0.0000 0.0000 0.0000 0.0000 0.0000 0.1602 0.0543 0.0370 0.0000 0.0000 0.0000 0.0000 0.0000 0.1291 0.0375 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000
18 0.2353 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2022 0.0597 0.0000 0.0391 0.0000 0.1010 0.0000 0.0560 0.0830 0.0000 1.0000 0.0000 0.0000 0.2176
19 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0739 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000
20 0.0000 0.0000 0.0000 0.0000 0.0000 0.2273 0.0737 0.2066 0.0000 0.0000 0.0000 0.0000 0.0000 0.2894 0.2090 0.0000 0.0000 0.0000 0.0000 1.0000 0.0650
21 0.2182 0.0000 0.0000 0.0000 0.0000 0.1699 0.0517 0.1055 0.0000 0.0000 0.0000 0.0000 0.0000 0.1543 0.0748 0.0000 0.0000 0.2176 0.0000 0.0650 1.0000

BIN
static/malloc-std.png Executable file

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.7 KiB

View File

@ -0,0 +1,77 @@
# Copyright 2019 sillydan1
# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
# The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
# Sentence similarity graph plotter
# uncomment this for manual operation of the dataset plotted
# my_dataset = "./sentence_similarities.csv" # ARG1
set parametric
set size square
# Styling
set pointsize 7.5
set style fill solid 1.0 border rgb 'grey30'
set style line 1 lc rgb 'black' pt 6 lw 0.5
# Basically a one-dimensional circular coordinate system
fx(t) = cos(t)
fy(t) = sin(t)
rownum = floor(system("wc -l ".my_dataset."")) +1
coord(k) = (k/real(rownum))*(2*pi)
fxx(t) = cos(coord(t))
fyy(t) = sin(coord(t))
set trange [0:2*pi-(coord(1.0))]
set sample rownum
set noborder
unset tics
set xrange [-1.2:1.2]
set yrange [-1.2:1.2]
set title "Sentence inter-similarity graph"
set multiplot
refloptimization = 0
do for [i = 0:rownum-1] {
do for [j = refloptimization:rownum-1] {
if (i != j) {
# Get how many columns there are in the dataset.
arrwidth = real(system("awk 'FNR == ".(i+1)." {print $".(j+1)."}' ".my_dataset.""))
if (arrwidth > 0.0) {
bubblerad = 0.125
x1 = fxx(i)
y1 = fyy(i)
x2 = fxx(j)
y2 = fyy(j)
dvx = x2-x1
dvy = y2-y1
dvl = sqrt((dvx ** 2) + (dvy ** 2))
x1 = x1 + (dvx/dvl)*bubblerad
y1 = y1 + (dvy/dvl)*bubblerad
x2 = x2 - (dvx/dvl)*bubblerad
y2 = y2 - (dvy/dvl)*bubblerad
# Overleaf's arrow-width rendering is pretty terrible,
# so we use a color-gradient to determine connection-strength.
if (arrwidth > 0.2) {
col = "#000000"
} else {
if (arrwidth < 0.1) {
col = "#B8B8B8"
} else {
col = "#E4E4E4"
}
}
set arrow "".i.j."" from x1,y1 to x2,y2 nohead lw 0.5 lc rgb col
#set label "H" at (fxx(j)-fxx(i)),(fyy(j)-fyy(i))
show arrow "".i.j.""
}
}
}
refloptimization = refloptimization + 1
}
# Plot the circles
plot '+' u (fx(t)):(fy(t)) w p ls 1 notitle
# Plot the sentence labels
plot '+' u (fx(t)):(fy(t)):(sprintf("s.%d",$0+1)) with labels notitle

File diff suppressed because it is too large Load Diff

After

Width:  |  Height:  |  Size: 80 KiB