feat: I found my old blog
It only had 2 posts, but whatever
This commit is contained in:
parent
23ea4544fd
commit
aed85865a9
3
Makefile
Normal file
3
Makefile
Normal file
@ -0,0 +1,3 @@
|
||||
demo:
|
||||
docker build -t wip .
|
||||
docker run --rm -it -p 8080:8080 wip hugo serve --bind 0.0.0.0 --port 8080
|
@ -19,3 +19,7 @@ docker run --rm -it -p 8080:8080 wip hugo serve --bind 0.0.0.0 --port 8080
|
||||
- [ ] how to securely "self-host" using a VPS, portainer and traefik
|
||||
- [ ] how to configure neomutt
|
||||
- [ ] how to securely host a mail server
|
||||
|
||||
### Old sillyblog
|
||||
- [x] Avr memory model
|
||||
- [x] similarity graph
|
||||
|
110
content/posts/avr-memory-model.md
Normal file
110
content/posts/avr-memory-model.md
Normal file
@ -0,0 +1,110 @@
|
||||
+++
|
||||
date = '2019-03-24'
|
||||
draft = false
|
||||
title = 'AVR Memory Model: The Practical Explanation'
|
||||
tags = ['avr', 'programming', 'memory']
|
||||
categories = ['technical']
|
||||
+++
|
||||
|
||||
There is A LOT of people that have already explained this, but I personally don't feel like going through billions of
|
||||
forum posts, with most of them just dying out somewhere in 2008 with no better answer than "I figured it out, thanks".
|
||||
|
||||
Regardless. BIG shoutout to the amazing people at [AVR Freaks](https://www.avrfreaks.net/). They are really cool.
|
||||
Seriously. Make a user and ask them about anything, and they'll help you.
|
||||
|
||||
## Disclaimer
|
||||
|
||||
I have only been debugging the memory usage of a specific ATMega chip. I don't know if other AVR chip-types use the
|
||||
same, but this explanation should be valid for all MCUs that the
|
||||
[avr-libc](http://www.nongnu.org/avr-libc/user-manual/index.html) package supports.
|
||||
|
||||
I also assume that GNU/Linux is being used on the development computer.
|
||||
|
||||
### Open-Source development tools
|
||||
|
||||
The [avr-gcc](http://www.nongnu.org/avr-libc/user-manual/pages.html) compiler chain is an open source effort to have
|
||||
C/C++ for AVR Atmel chips. They do provide some rudimentary C++ support, but there's no STL and the `new` and `delete`
|
||||
keywords are not implemented by default. Even purely virtual functions doesn't work out of the box.
|
||||
|
||||
**But don't fret!** There are ways to implement those features manually. See my [other]() post about getting the build
|
||||
environment up and running.
|
||||
|
||||
## The Memory Model
|
||||
|
||||
As the avr-libc developers [explain](http://www.nongnu.org/avr-libc/user-manual/malloc.html), there's typically not a
|
||||
lot of RAM available on most many devices and therefore it's very important to keep track of how much memory you are
|
||||
using.
|
||||
|
||||
{{< centered image="/malloc-std.png" >}}
|
||||
|
||||
All of these symbols `SP`, `RAMEND`, `__data_start`, `__malloc_heap_start`, etc. Can be modified in the compiler, but
|
||||
the picture above gives the default layout (for an ATMega128 MCU). It goes without saying, that if you don't have an
|
||||
external RAM chip, you won't be able to utilize the extra RAM space for that. Otherwise, the memory addresses are pretty
|
||||
straight forward: `0x0100 => 256` bytes is the start of the memory, `0x10FF => 4351` bytes is the end. If you're
|
||||
wondering where the RAM ends on your specific MCU, you can usually simply open the spec-sheet of the chip and see the
|
||||
amount of available memory is in it.
|
||||
For the [ATMega128](https://www.microchip.com/wwwproducts/en/ATMEGA128) that number is 4096 (`4351 - 256 = 4095` (the
|
||||
spec-sheet also counts the 0th byte)).
|
||||
|
||||
## The avr-libc Memory Allocators
|
||||
|
||||
Now for the juicy part. whenever you `malloc` something in your program, the allocator first writes a 2-byte *free-list
|
||||
entry* that tells the system how big your object is.
|
||||
|
||||
Example:
|
||||
|
||||
```cpp
|
||||
/* ... */
|
||||
// Allocate an array of 5 integers
|
||||
int* my_heap_object = static_cast<int*>(malloc(sizeof(int) * 5));
|
||||
/* ... */
|
||||
```
|
||||
|
||||
Assuming that the memory has been cleared on chip-startup, the above example ends up with the memory setup looking like
|
||||
this: (Don't mind the specifc memory addresses. If you're curious, you can try doing this, by attaching `avr-gdb` to a
|
||||
simulator or On Chip Debugger (OCD)).
|
||||
|
||||
```
|
||||
gdb: > x/16xb my_heap_object
|
||||
0x800100: 0a 00 00 00 00 00 00 00
|
||||
0x800108: 00 00 00 00 00 00 00 00
|
||||
```
|
||||
|
||||
The first bytes at address `0x800100` are `0a` and `00`. These bytes are the *free-list* entry and explains how "big"
|
||||
the object is. When reading this, we have to remember that the model is littleengine-based (meaning that the bytes are
|
||||
switched),
|
||||
so we actually have the value of `0x000a`, meaning `10` in decimal. This makes a lot of sense, since we allocated 5
|
||||
`int`s, that is of size 2 (16bit integers).
|
||||
|
||||
The memory dump shows 16 bytes in total, so the last 4 bytes displayed in the gdb example are not part of the object.
|
||||
However, if you look at the Memory Model picture again, you can see that the `__brkval` value points to the biggest
|
||||
memory address that has not been allocated. In our example, if you check where the `__brkval` points to after our
|
||||
allocation, we get:
|
||||
|
||||
```
|
||||
gdb: > __brkval
|
||||
$ 268
|
||||
```
|
||||
|
||||
268 in hexadecimal is `0x10c`, and if interpreted as an address we get `0x80010c`, which fits very well with our
|
||||
example, since it is exactly 12 bytes away from where the free-list entry of `my_heap_object` is located at.
|
||||
|
||||
When `free`-ing the object again, the deallocator looks at the free-list entry at the given address, and wipes the
|
||||
free-list entry. **This is why you should not free a dangling pointer**. Freeing something that is not really free-list
|
||||
entry *will* result in undefined behaviour, and I think we all know how bad **that** is. (Even though the AVR
|
||||
environment is actually very good at handling it. In my experience, it usually just crashes and starts over.)
|
||||
However, as
|
||||
[explained](http://www.nongnu.org/avr-libc/user-manual/group__avr__stdlib.html#gafb8699abb1f51d920a176e695ff3be8a) in
|
||||
the avrlibc documentation, freeing the `NULL` value, doesn't do anything. So remember to assign your free'd pointers to
|
||||
`NULL` afterwards.
|
||||
|
||||
## Wrapping up
|
||||
|
||||
The memory allocators of AVR can be very confusing and if you don't keep your thoughts straight when programming, you
|
||||
can very easily get yourself into a lot of trouble. Since STL is not available to avr-gcc programmers, we dont have our
|
||||
glorious smart pointers, so we should implement them ourselves (or use arduino's implementations). That might become a
|
||||
future blogpost.
|
||||
|
||||
Regardless, I hope this helps the lost souls that are trying to actually use these tools.
|
||||
|
||||
{{< centered image="/6616144.png" >}}
|
103
content/posts/bubble-graph.md
Normal file
103
content/posts/bubble-graph.md
Normal file
@ -0,0 +1,103 @@
|
||||
+++
|
||||
date = '2019-03-13'
|
||||
draft = false
|
||||
title = 'Plotting Circular Bubble Graph in LaTeX'
|
||||
tags = ['latex', 'programming']
|
||||
categories = ['technical']
|
||||
+++
|
||||
|
||||
When doing web-intelligence, you might need to visualize what's called inter-sentence similarity in a particular format.
|
||||
I personally haven't found an official name for these kinds of graphs, so I just simply call them Circular Bubble
|
||||
Graphs.
|
||||
|
||||
During a university project we needed such a graph in our report, and I got the idea of automatically plotting it
|
||||
through `gnuplot` and integrating it directly into our report with `gnuplottex`. You can see an example of the outcome
|
||||
of the script.
|
||||
|
||||

|
||||
|
||||
The script operates on a comma-seperated file (`.csv`). The data should be provided as a matrix of sentences assumed to
|
||||
be symmetric, with the cells containing a real number from 0 to 1, indicating the similarity between the column and the
|
||||
row. Because of this symmetric property, half of the matrix is ignored by the script. (It also ignores the diagonal,
|
||||
since sentence `23` will always have a maximum similarity to sentence `23`. It would also be hard to plot that line)
|
||||
|
||||
The whole script can be seen below, but you can also download it as a file
|
||||
[here](/sentence_similarity_graph.gnuplot). Make sure to set the `my_dataset` variable to your desired
|
||||
dataset. Example matrix can be downloaded [here](/example_similarities.csv).
|
||||
|
||||
```bash
|
||||
# Sentence similarity graph plotter
|
||||
# uncomment this for manual operation of the dataset plotted
|
||||
# my_dataset = "./sentence_similarities.csv" # ARG1
|
||||
set parametric
|
||||
set size square
|
||||
|
||||
# Styling
|
||||
set pointsize 7.5
|
||||
set style fill solid 1.0 border rgb 'grey30'
|
||||
set style line 1 lc rgb 'black' pt 6 lw 0.5
|
||||
|
||||
# Basically a one-dimensional circular coordinate system
|
||||
fx(t) = cos(t)
|
||||
fy(t) = sin(t)
|
||||
rownum = floor(system("wc -l ".my_dataset."")) +1
|
||||
coord(k) = (k/real(rownum))*(2*pi)
|
||||
fxx(t) = cos(coord(t))
|
||||
fyy(t) = sin(coord(t))
|
||||
|
||||
set trange [0:2*pi-(coord(1.0))]
|
||||
set sample rownum
|
||||
set noborder
|
||||
unset tics
|
||||
set xrange [-1.2:1.2]
|
||||
set yrange [-1.2:1.2]
|
||||
set title "Sentence inter-similarity graph"
|
||||
set multiplot
|
||||
refloptimization = 0
|
||||
do for [i = 0:rownum-1] {
|
||||
do for [j = refloptimization:rownum-1] {
|
||||
if (i != j) {
|
||||
# Get how many columns there are in the dataset.
|
||||
arrwidth = real(system("awk 'FNR == ".(i+1)." {print $".(j+1)."}' ".my_dataset.""))
|
||||
if (arrwidth > 0.0) {
|
||||
bubblerad = 0.125
|
||||
x1 = fxx(i)
|
||||
y1 = fyy(i)
|
||||
x2 = fxx(j)
|
||||
y2 = fyy(j)
|
||||
|
||||
dvx = x2-x1
|
||||
dvy = y2-y1
|
||||
dvl = sqrt((dvx ** 2) + (dvy ** 2))
|
||||
x1 = x1 + (dvx/dvl)*bubblerad
|
||||
y1 = y1 + (dvy/dvl)*bubblerad
|
||||
x2 = x2 - (dvx/dvl)*bubblerad
|
||||
y2 = y2 - (dvy/dvl)*bubblerad
|
||||
# Overleaf's arrow-width rendering is pretty terrible,
|
||||
# so we use a color-gradient to determine connection-strength.
|
||||
if (arrwidth > 0.2) {
|
||||
col = "#000000"
|
||||
} else {
|
||||
if (arrwidth < 0.1) {
|
||||
col = "#B8B8B8"
|
||||
} else {
|
||||
col = "#E4E4E4"
|
||||
}
|
||||
}
|
||||
|
||||
set arrow "".i.j."" from x1,y1 to x2,y2 nohead lw 0.5 lc rgb col
|
||||
#set label "H" at (fxx(j)-fxx(i)),(fyy(j)-fyy(i))
|
||||
show arrow "".i.j.""
|
||||
}
|
||||
}
|
||||
}
|
||||
refloptimization = refloptimization + 1
|
||||
}
|
||||
# Plot the circles
|
||||
plot '+' u (fx(t)):(fy(t)) w p ls 1 notitle
|
||||
|
||||
# Plot the sentence labels
|
||||
plot '+' u (fx(t)):(fy(t)):(sprintf("s.%d",$0+1)) with labels notitle
|
||||
```
|
||||
|
||||
{{< centered image="/6616144.png" >}}
|
21
static/example_similarities.csv
Executable file
21
static/example_similarities.csv
Executable file
@ -0,0 +1,21 @@
|
||||
1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0492 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2353 0.0000 0.0000 0.2182
|
||||
0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 1.0000 0.0000 0.1694 0.0510 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1186 0.0371 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 0.1694 0.0000 1.0000 0.2748 0.0286 0.2232 0.2697 0.1424 0.0000 0.0000 0.2443 0.2312 0.0000 0.1602 0.0000 0.0000 0.2273 0.1699
|
||||
0.0000 0.0000 0.0000 0.0510 0.0000 0.2748 1.0000 0.0480 0.0627 0.0761 0.0432 0.0000 0.0000 0.2907 0.1857 0.0000 0.0543 0.0000 0.0000 0.0737 0.0517
|
||||
0.0492 0.0000 0.0000 0.0000 0.0000 0.0286 0.0480 1.0000 0.0520 0.0000 0.0634 0.0000 0.0739 0.1664 0.1848 0.1117 0.0370 0.2022 0.0739 0.2066 0.1055
|
||||
0.0000 0.0000 0.0000 0.0000 0.0000 0.2232 0.0627 0.0520 1.0000 0.0000 0.0000 0.0000 0.0000 0.1581 0.0518 0.0000 0.0000 0.0597 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 0.0000 0.0000 0.2697 0.0761 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.2148 0.0595 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 0.0000 0.0000 0.1424 0.0432 0.0634 0.0000 0.0000 1.0000 0.0000 0.0000 0.1091 0.0304 0.0000 0.0000 0.0391 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0739 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.1010 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 0.1186 0.0000 0.2443 0.2907 0.1664 0.1581 0.2148 0.1091 0.0000 0.0000 1.0000 0.2803 0.0000 0.1291 0.0000 0.0000 0.2894 0.1543
|
||||
0.0000 0.0000 0.0000 0.0371 0.0000 0.2312 0.1857 0.1848 0.0518 0.0595 0.0304 0.0000 0.0000 0.2803 1.0000 0.0000 0.0375 0.0560 0.0000 0.2090 0.0748
|
||||
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1117 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0830 0.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 0.0000 0.0000 0.1602 0.0543 0.0370 0.0000 0.0000 0.0000 0.0000 0.0000 0.1291 0.0375 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000
|
||||
0.2353 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2022 0.0597 0.0000 0.0391 0.0000 0.1010 0.0000 0.0560 0.0830 0.0000 1.0000 0.0000 0.0000 0.2176
|
||||
0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0739 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000
|
||||
0.0000 0.0000 0.0000 0.0000 0.0000 0.2273 0.0737 0.2066 0.0000 0.0000 0.0000 0.0000 0.0000 0.2894 0.2090 0.0000 0.0000 0.0000 0.0000 1.0000 0.0650
|
||||
0.2182 0.0000 0.0000 0.0000 0.0000 0.1699 0.0517 0.1055 0.0000 0.0000 0.0000 0.0000 0.0000 0.1543 0.0748 0.0000 0.0000 0.2176 0.0000 0.0650 1.0000
|
|
BIN
static/malloc-std.png
Executable file
BIN
static/malloc-std.png
Executable file
Binary file not shown.
After Width: | Height: | Size: 2.7 KiB |
77
static/sentence_similarity_graph.gnuplot
Executable file
77
static/sentence_similarity_graph.gnuplot
Executable file
@ -0,0 +1,77 @@
|
||||
# Copyright 2019 sillydan1
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
|
||||
# The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
||||
# Sentence similarity graph plotter
|
||||
# uncomment this for manual operation of the dataset plotted
|
||||
# my_dataset = "./sentence_similarities.csv" # ARG1
|
||||
set parametric
|
||||
set size square
|
||||
|
||||
# Styling
|
||||
set pointsize 7.5
|
||||
set style fill solid 1.0 border rgb 'grey30'
|
||||
set style line 1 lc rgb 'black' pt 6 lw 0.5
|
||||
|
||||
# Basically a one-dimensional circular coordinate system
|
||||
fx(t) = cos(t)
|
||||
fy(t) = sin(t)
|
||||
rownum = floor(system("wc -l ".my_dataset."")) +1
|
||||
coord(k) = (k/real(rownum))*(2*pi)
|
||||
fxx(t) = cos(coord(t))
|
||||
fyy(t) = sin(coord(t))
|
||||
|
||||
set trange [0:2*pi-(coord(1.0))]
|
||||
set sample rownum
|
||||
set noborder
|
||||
unset tics
|
||||
set xrange [-1.2:1.2]
|
||||
set yrange [-1.2:1.2]
|
||||
set title "Sentence inter-similarity graph"
|
||||
set multiplot
|
||||
refloptimization = 0
|
||||
do for [i = 0:rownum-1] {
|
||||
do for [j = refloptimization:rownum-1] {
|
||||
if (i != j) {
|
||||
# Get how many columns there are in the dataset.
|
||||
arrwidth = real(system("awk 'FNR == ".(i+1)." {print $".(j+1)."}' ".my_dataset.""))
|
||||
if (arrwidth > 0.0) {
|
||||
bubblerad = 0.125
|
||||
x1 = fxx(i)
|
||||
y1 = fyy(i)
|
||||
x2 = fxx(j)
|
||||
y2 = fyy(j)
|
||||
|
||||
dvx = x2-x1
|
||||
dvy = y2-y1
|
||||
dvl = sqrt((dvx ** 2) + (dvy ** 2))
|
||||
x1 = x1 + (dvx/dvl)*bubblerad
|
||||
y1 = y1 + (dvy/dvl)*bubblerad
|
||||
x2 = x2 - (dvx/dvl)*bubblerad
|
||||
y2 = y2 - (dvy/dvl)*bubblerad
|
||||
# Overleaf's arrow-width rendering is pretty terrible,
|
||||
# so we use a color-gradient to determine connection-strength.
|
||||
if (arrwidth > 0.2) {
|
||||
col = "#000000"
|
||||
} else {
|
||||
if (arrwidth < 0.1) {
|
||||
col = "#B8B8B8"
|
||||
} else {
|
||||
col = "#E4E4E4"
|
||||
}
|
||||
}
|
||||
|
||||
set arrow "".i.j."" from x1,y1 to x2,y2 nohead lw 0.5 lc rgb col
|
||||
#set label "H" at (fxx(j)-fxx(i)),(fyy(j)-fyy(i))
|
||||
show arrow "".i.j.""
|
||||
}
|
||||
}
|
||||
}
|
||||
refloptimization = refloptimization + 1
|
||||
}
|
||||
# Plot the circles
|
||||
plot '+' u (fx(t)):(fy(t)) w p ls 1 notitle
|
||||
|
||||
# Plot the sentence labels
|
||||
plot '+' u (fx(t)):(fy(t)):(sprintf("s.%d",$0+1)) with labels notitle
|
1562
static/sentence_similarity_graph_example.svg
Executable file
1562
static/sentence_similarity_graph_example.svg
Executable file
File diff suppressed because it is too large
Load Diff
After Width: | Height: | Size: 80 KiB |
Loading…
x
Reference in New Issue
Block a user