It's my particles in a box!

You have three particles inside a unit square that all repel one another. The energy between each pair of particles is $1/r,$ where $r$ is the distance between them. To be clear, the particles can be anywhere inside the square or on its perimeter. The total energy of the system is the sum of the three pairwise energies among the particles.

What is the minimum energy of this system, and what arrangement of the particles produces it?

If we only had two particles, then the furthest away that the particles could be (and hence the minimal total energy arrangment), would be opposite corners of the unit square, e.g., one at $(0,0)$ and the other at $(1,1).$ In this configuration, the total energy would be $E_2 = \frac{1}{\sqrt{2}}.$ So let's assume that we have two of the three particles there and then see where the other particle would end up.

Assume that the third particle is at the point $X=X(a,b)$ for some $0 \leq a, b \leq 1.$ Then the total energy of the system would be

$$E = E(a,b) = \frac{1}{\sqrt{2}} + \frac{1}{\sqrt{a^2+b^2}} + \frac{1}{\sqrt{(1-a)^2 + (1-b)^2}}.$$ Taking derivatives gives $$\nabla E = \begin{pmatrix} -\frac{a}{(a^2 + b^2)^{3/2}} + \frac{1-a}{\left( (1-a)^2 + (1-b)^2 \right)^{3/2}} \\ -\frac{b}{(a^2 + b^2)^{3/2}} + \frac{1-b}{\left( (1-a)^2 + (1-b)^2 \right)^{3/2}}\end{pmatrix}.$$ The only analytic root of the gradient mapping is $a=b=\frac{1}{2},$ which would give a total energy of $E(\frac{1}{2},\frac{1}{2}) = \frac{1}{\sqrt{2}} + 2 \frac{2}{\sqrt{2}} = \frac{5}{\sqrt{2}},$ but this is a saddle point and we can do better by checking what happens if $X$ were along the perimeter of the unit square.

Assume without loss of generality and by availing ourselves of symmetry, that $b = 0$ and we just have $0 \leq a \leq 1.$ In this case, we have

$$\tilde{E}(a) = E(a,0) = \frac{1}{\sqrt{2}} + \frac{1}{a} + \frac{1}{\sqrt{1 + (1-a)^2}}$$ which has derivative $$\tilde{E}^\prime = -\frac{1}{a^2} + \frac{1-a}{\left(1 + (1-a)^2\right)^{3/2}} = \frac{a^2(1-a) - \left(a^2 - 2a+2\right)^{3/2}}{a^2\left(a^2 - 2a+2\right)^{3/2}}.$$ Since $$a^2(1-a) \leq \frac{4}{27} \lt 1 \leq \left(a^2 - 2a + 2\right)^{3/2},$$ for all $a \gt 0,$ we have $\tilde{E}^\prime \lt 0$ for all $a \gt 0,$ so the lowest possible value is when $a^* = 1,$ that is when $X^* = (1,0),$ and the total energy is at its lowest $$E^* = \tilde{E}(1) = \frac{1}{\sqrt{2}} + 2 \approx 2.70710678119....$$ There are other possible arrangements of the particles that obtain the same minimal energy. Namely, any of the four right isoceles triangle formed by choosing three out of the four vertices of the unit square will give this minimal energy configuration.