Random forest classification in Julia

Below is example code for fitting and evaluating a linear regression and random forest classifier in Julia. I've added the linear regression as a baseline for the random forest. The models are evaluated on a mock variable U generated from two distributions, namely

 $\begin{aligned} d_1 &= \text{Normal}(10, 2) \: \: \text{and} \\ d_2 &= \text{Normal}(12, 2), \end{aligned}$

The random variable V is just noise meant to test the classifier, generated via

2021-01-21

Random forest, Shapley values and multicollinearity

Linear statistical models are great for many use-cases since they are easy to use and easy to interpret. Specifically, linear models can use features (also known as independent variables, predictors or covariates) to predict an outcome (also known as dependent variables).

In a linear model, a higher coefficient for a feature, the more a feature played a role in making a prediction. However, when variables in a regression model are correlated, these conclusions don't hold anymore.

2020-12-16

GitHub and GitLab commands cheatsheet

Both GitHub and GitLab provide shortcuts for interacting with the layers they have built on top of Git. These shortcuts are a convenient and clean way to interact with things like issues and PRs. For instance, using Fixes #2334 in a commit message will close issue #2334 automatically when the commit is applied to the main branch. However, the layers on top of Git differ between the two, and therefore the commands will differ as well. This document is a cheatsheet for issue closing commands; I plan to add more of these commands over time.

2020-11-29

Design cheatsheet

I like to complain that design can distract from the main topic and is therefore not important. However, design is important. If your site, presentation or article looks ugly, then you already are one step behind in convincing the audience. The cheatsheet below can be used to quickly fix design mistakes.

Colors

Suprisingly, you should Never Use Black. Instead you can use a colors which are near black. For example:

Tint	HTML color code	Example text
Pure black	#000000	~~~ Lorem ipsum dolor sit amet ~~~
Grey	#4D4D4D	~~~ Lorem ipsum dolor sit amet ~~~
Green	#506455	~~~ Lorem ipsum dolor sit amet ~~~
Blue	#113654	~~~ Lorem ipsum dolor sit amet ~~~
Pink	#564556	~~~ Lorem ipsum dolor sit amet ~~~

2020-11-14

Frequentist and Bayesian coin flipping

To me, it is still unclear what exactly is the difference between Frequentist and Bayesian statistics. Most explanations involve terms such as "likelihood", "uncertainty" and "prior probabilities". Here, I'm going to show the difference between both statistical paradigms by using a coin flipping example. In the examples, the effect of showing more data to both paradigms will be visualised.

Generating data

Lets start by generating some data from a fair coin flip, that is, the probability of heads is 0.5.

import CairoMakie

using AlgebraOfGraphics: Lines, Scatter, data, draw, visual, mapping
using Distributions
using HypothesisTests: OneSampleTTest, confint
using StableRNGs: StableRNG

2020-11-07

Installing NixOS with encryption on a Lenovo laptop

In this post, I walk through the steps to install NixOS on a Lenovo Yoga 7 with an encrypted root disk. This tutorial is mainly based on the tutorial by Martijn Vermaat and comments by @ahstro and dbwest.

USB preparation

Download NixOS and figure out the location of the USB drive with lsblk. Use the location of the drive and not the partition, so /dev/sdb instead of /dev/sdb1. Then, prepare the USB with

2020-11-04

The logit and logistic functions

Linear regression works on real numbers \mathbb{R}, that is, the input and output are in \mathbb{R}. For probabilities, this is problematic because the linear regression will happily give a probability of -934, where we know that probabilities should always lie between 0 and 1. This is only by definition, but it is an useful definition in practice. Informally, the logistic function converts values from real numbers to probabilities and the logit function does the reverse.

2020-09-26

The principle of maximum entropy

Say that you are a statistician and are asked to come up with a probability distribution for the current state of knowledge on some particular topic you know little about. (This, in Bayesian statistics, is known as choosing a suitable prior.) To do this, the safest bet is coming up with the least informative distribution via the principle of maximum entropy.

This principle is clearly explained by Jaynes (1968): consider a die which has been tossed a very large number of times N. We expect the average to be 3.5, that is, we expect a distribution where P_n = \frac{1}{6} for each n, see the figure below.

2020-08-12

Writing effectively

According to McEnerney (2014), academics are trained to be poor writers. Eventually, they end up in his office and tell, while crying, that their careers might end soon. One reason why academics are poor writers is that they are expert writers. Expert writers are not experts in writing but are experts who write. An expert writer typically thinks via writing and they assume that this raw output is good enough for readers. However, it isn't good enough. For a start, expert writers have a worldview which differs from the readers' due to the writers' expertise. So, to avoid crying, McEnerney argues that writers should instead write to be valuable to the community of readers.

2020-07-29

Writing checklist

I keep forgetting lessons about writing. After writing a text, my usual response is to declare it as near perfect and never look at it again. In this text, I will describe a checklist, which I can use to quickly debunk the declaration. I plan to improve this checklist over time. Hopefully, text which passes the checklist in a few dozen years from now will, indeed, be near perfect.

The list is roughly ordered by importance. The text should:

Ensure that the writing is valuable to the community of readers.
Be simple (Adams, 2015) or be made as simple as possible, but not simpler. This is also known as Occam's razor, kill your darlings or the KISS principle.
Be polite, that is, not contain a career limiting move. For example, do not "write papers proclaiming the superiority of your work and the pathetic inadequacy of the contributions of A, B, C, ..." (Wadge 2020).
Be consistent. For example, either use the Oxford comma in the entire text or do not use it at all.
Avoid misspellings.
Avoid comma splices.
Place the object before the action, so write "the boy hit the ball" instead of "the ball was hit by the boy".
Flow naturally; just like a normal conversation. This is, for me, contradictory to writing when programming.
Provide a high-level overview of the text. This can be a summary, abstract, a few sentences in the introduction or a combination of these.
Prefer common collocations. A list of common collocations is The Academic Collocation List.
Use simple verbs, for example, prefer "stop" over "cease to move on" or "do not continue".
Avoid dying metaphors such as "stand shoulder to shoulder with" (Orwell, 1946). Metaphors aim to "assist thought by evoking a visual image" (Orwell, 1946). Dying metaphors do not evoke such an image anymore due to overuse (Orwell, 1946).
Avoid pretentious diction such as dressing up simple statements, inappropriate adjectives and foreign words and expressions (Orwell, 1946). For example, respectively "effective", "epic" and "status quo" (Orwell, 1946).
Avoid meaningless words, that is, words for which no clear definition exists. For example, "democracy" and "freedom" have "several different meanings which cannot be reconciled with one another" (Orwell, 1946).